UNCLASSIFIED 


QUALITATIVE  ASSESSMENT  REPORT 
PORTABLE  FLIGHT  PLANNING  SOFTWARE  (PFPS) 

September  1997 
by 

Capt  R.  S.  Barr  USMC 
Project  Action  Officer 
Assault  Support  Department 

MaJ  M.  T.  Sampson  USMC 
Coordinating  Officer 
ADT&E  Department 

for 

MaJ  J.  S.  Bennett  USMC 
PMA-233,  NAVAIRSYSCOM 

Completion  Date:  25  February  1998 

MARINE  AVIATION  WEAPONS  AND  TACTICS  SQUADRON  ONE 

MCAS  YUMA,  AZ 

DISTRIBUTION  STATEMENT:  Approved  for  public  release;  distribution  unlimited.  Requests  for  copies  of 
this  document  shall  be  referred  to  Commanding  Officer,  MAWTS-1 ,  Attn:  ADT&E,  Box  99200,  Yuma,  AZ 
85369-9200. 

DESTRUCTION  NOTICE:  Destroy  by  any  method  available. 

DTIC  QUAmT  rKSPEOTSP  9 

UNCLASSIFIED 


19980319  056 


_ REPORT  DOCUMENTATION  PAGE _  Form  Approved  QMB  No.  0704 

Public  rqwrting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  revierving 
instruction,  searching  existing  data  sources,  gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of 
information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for 
reducing  this  burden,  to  Washington  headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway, 
Suite  1204,  Arlington,  VA  222024302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188)  Washington 
DC  20503.’  _ _ _ _ _ _ _ _ _ 

I.  AGENCY  USE  ONLY  (Leave  blank)  1 2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

February  1998 _ Qualitative  Assessment,  Sep  97-Feb  98 _ 

4.  TITLE:  Qualitative  Assessme'nt  of  Portable  Flight  Planning  Software  (PFPS) _ 5.  FUNDING  NUMBERS:  NA 

6.  AUTHOR(S)  Barr,  Robert  S.;  Sampson,  Matthew  T. _ _ _ _ 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES)  8.  PERFORMING  ORGANIZATION 

Marine  Aviation  Weapons  and  Tactics  Squadron  One.  Box  99200  REPORT  NUMBER 

Yuma.  AZ  85369-9200 _ _ _ _ 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES)  10.  SPONSORING/MONTTORING 

Major  Steve  Bennett,  PMA  233,  NAVAIRSYSCOM,  NAS  Pataxent  River.  MD  20702  |  AGENCY  REPORT  NUMBER _ 

II.  SUPPLEMENTARY  NOTES:  The  views  expressed  in  this  report  are  those  of  MAWTS-1  and  do  not  reflect  the  official  policy  or 

position  of  the  Department  of  Defense  or  the  U.S.  Government. _ _ _ _ _ _ 

12a.  DISTRIBUTION/A VAlLABILrrY  STATEMENT:  Approved  for  public  release;  12b.  DISTRIBUTION  CODE: 

distribution  unlimited.  ^ 

13.  ABSTRACT:  With  the  exponential  increase  in  computer  capabilities  in  the  last  ten  years,  the  Department  of  Defense  and  the  Department 
of  the  Navy  (DoN)  has  sought  to  equip  Navy  and  Marine  forces  with  more  capable  and  compact  Automated  Mission  Planning  Systems  (AMPS). 
Despit  these  efforts,  there  currently  is  no  single  AMPS  which  satisfies  the  AMPS  requirements  for  all  DoN  aircraft.  ThCTe  are,  however, 
numerous  AMPS’s  being  employed  to  fill  the  shortfall.  One  interim  fix  being  explored  by  DoN  is  Portable  Flight  Planning  Software  (PFPS). 
PFPS  is  a  PC-based  AMPS,  developed  by  the  Air  Force  and  Georgia  Tech  Research  Institute,  which  presents  the  Combat  Flight  Plarmir^ 
Software  (CFPS)  interactive  time,  distance,  heading,  and  fuel  card  and  the  Falcon  View  mapping  toolkit  simultaneously.  Other  capabilities 
include  the  ability  to  interface  with  digital  maps,  imagery,  terrain  elevation  data  and  aeronautical  flight  information  files.  Upon  request  frotn 
PMA-233,  a  Qualitative  Asessment  (QA)  of  PFPS  was  conducted  by  Marine  Aviation  Weapons  and  Tactics  Squadron  One  (MAWTS-1)  during 
Weapons  and  Tactics  Instructor  (WTI)  Course  1-98.  Approximately  forty  experienced  aviators  from  the  Marine  Corps  and  Air  Force 
participated  in  the  assessment.  The  asessment  consisted  of  classrom  training,  computer  lab,  and  self  paced  study.  Respondent  opinions  were 
capured  in  a  single  survey  administered  three  times  during  the  course.  Respondents  were  surveyed  on  their  background  information,  their 
computer  and  AMPS  experience,  the  adequacy  of  PFPS  training  received,  the  difficulty  of  using  PFPS  Human  Machine  Interfaces  (HMI),  the 
adequacy  of  PFPS  documentation,  the  compatability  of  PFPS  with  various  mission  planning  requirements,  the  reliability  of  PFPS,  the 
maintainability  of  PFPS,  and  the  tactical  impact  and  time  savings  realized  while  using  PFPS.  Also,  proposed  functionality  enhancement  valididy 
and  functionality  enhancement  ranking  was  asessed.  PFPS  was  very  well  received  by  the  respondents  as  an  extremely  valuable  planning  tool 
and  it  was  quickly  assimilated  into  their  mission  planning  tasks.  After  reviewing  the  data  from  the  surveys  and  coupling  it  with  observations, 
human  factors,  compatability,  reliability,  maintainability,  and  tactical  impact/time  savings  of  PFPS  were  all  noted  for  possessing  enhancing 
characteristics.  Slight  deficiences  in  training,  hnman  factors,  documentation,  maintainability,  and  other  areas  were  noted.  Based  on  the 
asessment  results  and  observations,  the  Project  Action  Officer  recommends  the  following  actions  to  CNO  N62,  CNO  N880  and  PMA-233  as 
appropriate:  ( 1 )  pfovitie  rapid  dissemination  of  the  most  current  version  of  PFPS  to  all  DoN  squadrons;  (2)  plan  and  provide  for  rapid 
implementation  of  PFPS  Mission  Data  Loader  Receptacle  (MDLR)  hardware  and  software  upgrades  for  all  existing  and  future  DoN  mission 
planning  computers;  (3)  fund  PFPS  Serpentine  (Circumlinear)  route  drawing  tool  capability,  Map  Scan  capability  and  Ae  Hover  Tool;  (4) 
augment  existing  fleet  AMPS  computer  hardware  with  limited  laptop  and  desktop  computer  buys;  (5)  fund  the  capability  to  create  a  single 
mission  folder  to  hold  multiple  route,  threat  and  drawing  overlays;  (6)  fund  rapid  correction  of  all  deficiencies  with  PFPS  noted  in  this 
assessment  report,  and  (7)  fund  the  remaining  Flight  Performance  Modules  for  all  aircraft  in  the  Naval  Aviation  Inventory  and  any  aircraft 
currently  under  development  (i.e.,  V-22).  The  Project  Action  Qfficer  also  recommends  that  follow-on  AMPS’s  (TAMPS  7.0/JMPS)  be  modeled 

after  PFPS  and  that  future  AMPS  architecture  be  aligned  with  the  computer  capabilities  of  the  users,  as  presented  in  this  assessment. _ 

14.  SUBJECT  TERMS  15.  NUMBER  QF  PAGES 

PFPS,  Mission  Planning  Systems,  TAMPS,  Automated  Mission  Planning,  MDLR  - 62 - 

_ _ _ _ _ _ 16.  PRICE  CQDE _ 

17  SECURITY  CLASSIFICATION  OF  1 18.  SECURITY  |19.  SECURITY  20.  LIMITATION  OF  ABSTRACT 

report  classification  of  this  CLASSIFICATION 

page  of  abstract  Unlimited 


14.  SUBJECT  TERMS 

PFPS,  Mission  Planning  Systems,  TAMPS,  Automated  Mission  Planning,  MDLR 

17.  SECURITY  CLASSIFICATION  OF 

18.  SECURITY 

19.  SECURITY 

REPORT 

CLASSIFICATION  OF  THIS 

CLASSIFICATION 

PAGE 

OF  ABSTRACT 

Unclassified 

Unclassified 

Unclassified 

c 

NSN  7540-01-280-5500 


KioQOAiarrrais®''’"®'' 


Prescribed  by  ANSI  Std.  239-18 


QUALITATIVE  ASSESSMENT  REPORT 


PORTABLE  FLIGHT  PLANNING  SOFTWARE  (PFPS) 

Marine  aviation  weapons  and  tactics  squadron  one 

FEBRUARY  1998 

SUBMITTED  BY:  _ _ 

Robert  S.  Barr 


Matthew  T.  Sampsoi 


APPROVED  BY: 


Keith  J.  Stalder 
Commanding  Officer 


EXECUTIVE  SUMMARY 


With  the  exponential  increase  in  computer  capabilities  in  the  last  ten  years,  the  Department  of 
Defense  and  the  Department  of  the  Navy  (DoN)  has  sought  to  equip  Navy  and  Marine  forces  with 
more  capable  and  compact  Automated  Mission  Planning  Systems  (AMPS).  Despit  these  efforts, 
there  currently  is  no  single  AMPS  which  satisfies  the  AMPS  requirements  for  all  DoN  aircraft. 
There  are,  however,  numerous  AMPS’s  being  employed  to  fill  the  shortfall.  One  interim  fix  being 
explored  by  DoN  is  Portable  Flight  Planning  Software  (PFPS).  PFPS  is  a  PC-based  AMPS, 
developed  by  the  Air  Force  and  Georgia  Tech  Research  Institute,  which  presents  the  Combat 
Flight  Planning  Software  (CFPS)  interactive  time,  distance,  heading,  and  fuel  card  and  the  Falcon 
View  mapping  toolkit  simultaneously.  Other  capabilities  include  the  ability  to  interface  with  digital 
maps,  imagery,  terrain  elevation  data  and  aeronautical  flight  information  files. 

Upon  request  from  PMA-233,  a  Qualitative  Asessment  (QA)  of  PFPS  was  conducted  by 
Marine  Aviation  Weapons  and  Tactics  Squadron  One  (MAWTS-1)  during  Weapons  and  Tactics 
Instructor  (WTI)  Course  1-98.  Approximately  forty  experienced  aviators  from  the  Marine  Corps 
and  Air  Force  participated  in  the  assessment.  The  asessment  consisted  of  classrom  training, 
computer  lab,  and  self  paced  study.  Respondent  opinions  were  capured  in  a  single  survey 
administered  three  times  during  the  course.  Respondents  were  surveyed  on  their  background 
information,  their  computer  and  AMPS  experience,  the  adequacy  of  PFPS  training  received,  the 
difficulty  of  using  PFPS  Human  Machine  Interfaces  (HMI),  the  adequacy  of  PFPS  documentation, 
the  compatability  of  PFPS  with  various  mission  planning  requirements,  the  reliability  of  PFPS,  the 
maintainability  of  PFPS,  and  the  tactical  impact  and  time  savings  realized  while  using  PFPS. 

Also,  proposed  functionality  enhancement  valididy  and  functionality  enhancement  ranking  was 
asessed. 

PFPS  was  very  well  received  by  the  respondents  as  an  extremely  valuable  planning  tool  and  it 
was  quickly  assimilated  into  their  mission  planning  tasks.  After  reviewing  the  data  from  the 
surveys  and  coupling  it  with  observations,  human  factors,  compatability,  reliability,  maintainability, 
and  tactical  impact/time  savings  of  PFPS  were  all  noted  for  possessing  enhancing  characteristics. 
Slight  deficiences  in  training,  human  factors,  documentation,  maintainability,  and  other  areas 
were  noted. 

Based  on  the  asessment  results  and  observations,  the  Project  Action  Officer  recommends  the 
following  actions  to  CNO  N62,  CNO  N880  and  PMA-233  as  appropriate:  (1)  provide  rapid 
dissemination  of  the  most  current  version  of  PFPS  to  all  DoN  squadrons;  (2)  plan  and  provide  for 
rapid  implementation  of  PFPS  Mission  Data  Loader  Receptacle  (MDLR)  hardware  and  software 
upgrades  for  all  existing  and  future  DoN  mission  planning  computers;  (3)  fund  PFPS  Serpentine 
(Circumlinear)  route  drawing  tool  capability.  Map  Scan  capability  and  the  Hover  Tool;  (4)  augment 
existing  fleet  AMPS  computer  hardware  with  limited  laptop  and  desktop  computer  buys;  (5)  fund 
the  capability  to  create  a  single  mission  folder  to  hold  multiple  route,  threat  and  drawing  overlays; 
(6)  fund  rapid  correction  of  all  deficiencies  with  PFPS  noted  in  this  assessment  report,  and  (7) 
fund  the  remaining  Flight  Performance  Modules  for  all  aircraft  in  the  Naval  Aviation  Inventory  and 
any  aircraft  currently  under  development  (i.e.,  V-22).  The  Project  Action  Officer  also  recommends 
that  follow-on  AMPS’s  (TAMPS  7.0/JMPS)  be  modeled  after  PFPS  and  that  future  AMPS 
architecture  be  aligned  with  the  computer  capabilities  of  the  users,  as  presented  in  this 
assessment. 
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1.0  BACKGROUND  With  the  exponential  increase  in  computer  capabilities  in  the  last  ten 
years,  the  Department  of  Defense  and  the  Department  of  the  Navy  (DoN)  has  sought  to  equip 
Navy  and  Marine  forces  with  more  capable  and  compact  Automated  Mission  Planning  Systems 
(AMPS).  One  of  the  first  AMPSs,  procured  in  1986  by  the  Navy,  was  the  Tactical  Automated 
Mission  Planning  System  (TAMPS).  TAMPS  was  originally  purchased  by  the  Navy  without  going 
through  normal  acquisition  channels.  However,  since  the  establishment  of  the  program  office  in 
1990,  an  Operational  Requirements  Document  (ORD)  and  a  Test  and  Evaluation  Master  Plan 
(TEMP)  for  DoN  AMPSs  have  been  developed.  The  ORD  for  DoN  AMPSs  is  currently  under 
revision  by  the  program  sponsor,  N-62. 

TAMPS,  through  its  many  software  versions  and  hardware  configurations,  has  filled  the 
mission  planning  needs  of  the  FA-18  community  for  the  past  1 1  years.  Although  many  attempts 
have  been  made  to  make  TAMPS  useable  tool  for  Assault  Support  Aircraft  (all  helicopters  and 
KC-130S),  this  system  does  not  currently  have  a  true  mission  planning  capability  for  those 
platforms. 

In  an  effort  to  produce  a  TAMPS  version  that  supports  all  Navy/Marine  air/ground  assets  as 
well  as  one  which  complies  with  Joint  AMPS  interoperability  guidelines,  PMA-233  has  ordered  the 
last  TAMPS  functionality  release  (TAMPS  6.2,  due  in  1998)  while  focusing  on  a  complete 
reengineering  of  TAMPS  to  be  released  by  2003  (TAMPS  7.0). 

In  order  to  provide  a  partial  solution  for  the  AMPS  needs  of  all  air  and  ground  communities, 
PMA-233  has  procured  the  Portable  Flight  Planning  Software  (PFPS)  from  the  46th  Test 
Squadron  at  Eglin  AFB,  FL.  In  June  of  1997,  MAWTS-1  was  asked  by  PMA-233  to  conduct  a 
Qualitative  Assessment  (QA)  of  this  software  during  WT1 1-98.  The  C.O.  of  MAWTS-1  approved 
the  assessment  plan  in  September  1997.  A  QA  on  PFPS  was  conducted  at  MAWTS-1  during  the 
Weapons  and  Tactics  Instructor  (WTI)  1-98  class. 

2.0  PURPOSE  The  purpose  of  this  Qualitative  Assessment  was  to  assess  the  capabilities  of 
PFPS  as  an  AMPS  and  to  survey  fleet  representatives  attending  WT1 1-98  on  additional 
capabilities  they  would  like  to  see  in  TAMPS  7.0  or  any  future  AMPSs  as  they  are  fielded. 

3.0  DESCRIPTION  OF  SOFTWARE. 

3.1  Basic  Aircraft  Not  applicable. 

3.2  Assessment  Aircraft  Modification/lnstaimentation  Not  applicable. 

3.3  Assessment  Item.  PFPS  Version  3.0,  Phase  2A  (beta),  is  a  joint,  Air  Force  and  Georgia 
Tech  Research  Instituted,  developed,  government-owned,  PC-based,  AMPS  that  integrates 
FalconView  digital  mapping  software.  Combat  Flight  Planning  Software  (CFPS)  and  other 
software  to  provide  the  user  with: 


-  Windows  95  /  NT-4.0  Graphical  User  Interface  (GUI) 

-  Point  and  click  route  editing 

-  Selectable  map  scales 
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-  Unclassified  (Jane’s  type)  threat  plotting  (RTM,  MERA) 

-  Customized  kneeboard  card,  map  products  and  artwork 

-  Interoperability  with  other  AMPSs 

-  NIMA  standard  world  maps,  DAFIF,  CMS  DIED*  and  other  interfaces 


Figure  3.1  is  a  diagram  depicting  PFPS  product  modules  that  'plug-in'  to  the  'core'  software, 
each  module  is  shaded  according  to  its  developer.  These  modules  can  be  added,  replaced  or 
removed  as  required.  Figure  3.2  is  a  wire  diagram  depicting  the  relationships  between  the  central 
hub  (Route  Server)  and  all  of  the  product  modules.  Also  depicted  are  the  interrelationships 
between  the  various  product  modules.  Figure  3.3  is  a  basic  depiction  of  'how  PFPS  works', 
focusing  on  some  of  the  modules  users  may  access  more  frequently. 


PFPS  Products 


Portable  Flight 


Planning  Software 


—  Legend  — i 

n 

Not  46TS  Developed 
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46TS 
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CFPS  -  Combat  Flight  Planning 
CAPS  -  Combat  Air  Drop  Planning 
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Figure  3.1.  PFPS  Products.  A  conceptualization  of  how  product  modules,  shaded 
according  to  their  developer,  'plug-in'  to  the  core  software. 


*  NIMA  DTED  incorporation  is  expected  as  an  interim  release  between  PFPS  3.0  and  3.1 
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PFPS  Architecture 


Figure  3.2.  PFPS  Architechture.  Depicted  is  the  relationship  between  the  central  hub 
(Route  Server)  and  ail  other  product  modules.  Also  depicted  are  the  product  module 
interrelationships.  Annotations  provide  basic  descriptions  of  the  module's  function. 

PFPS  Architecture 


(How  Does  this  Work?) 


Figure  3.3  PFPS  Architecture.  Depicted  is  a  diagram  of  basically  'how  PFPS  works' 
illustrating  interactions  between  some  of  the  major  product  modules. 
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Appendix  A  is  a  brochure  on  PFPS  prepared  by  the  46th  Test  Squadron.  The  brochure  is 
written  to  highlight  the  capabilities  PFPS  provides  to  its  original  customer,  the  Air  Force,  however, 
it  does  not  contain  information  on  the  upgrades  in  the  PFPS  version  that  will  be  used  in  this 
assessment. 

4.0  SCOPE  OF  ASSESSMENT 

4.1  Assessment  and  Assessment  Conditions.  This  assessment  involved  training  students  on 
PFPS,  surveying  them  on  their  knowledge  and  opinions  of  current  AMPSs  and  the  suitability  of 
PFPS,  allowing  them  to  become  more  familiar  with  the  software  and  then  surveying  them  two 
more  times  about  PFPS  and  what  they  would  like  to  see  in  future  AMPSs. 

4.2  Assessment  Envelope.  This  assessment  was  conducted  solely  during  WTM -98.  Training 
and  surveying  was  conducted  during  scheduled  times  (Table  2).  Self-paced  training  by  students 
was  conducted  on  a  not-to-interfere  basis  with  other  WTI  evolutions. 

4.3  Flight  Clearance.  Since  PFPS  mission  data  loader  capabilities  was  not  available,  a  flight 
clearance  for  this  assessment  was  not  applicable. 

4.4  Assessment  Loadings.  PFPS  Version  3.0,  Phase  2A  (beta)  was  the  only  software  loaded 
on  computers  involved  in  this  assessment.  Students  and  instructors  were  encouraged  to  install 
only  this  version  on  any  personal  or  community  computers  they  may  practice  on. 

4.5  Assessment  Configurations/Profiles.  PFPS  Version  3.0,  Phase  2C  (beta)  was  installed 
on  a  three-computer  suite,  located  in  the  mission  planning  room  (S-2  Ready  Room)  and 
consisting  of  one  desktop  200  MHz  Pentium  PC  and  two  notebook  200  MHz  Pentium  PCs.  This 
suite  was  linked  by  way  of  a  junction  box  to  a  single  Hewlett  Packard  870CXI  DeskJet  color 
printer.  Since  the  software  includes  only  the  FA-18  Flight  Performance  Module  in  its  aircraft 
selection  menu,  baseline  flight  plan  files  were  created  and  saved  in  CFPS  for  the  other 
type/model/series  aircraft.  This  'work-around'  enabled  users  to  open  an  aircraft-specific  flight  plan 
with  its  'homeplate'  (first  waypoint)  located  at  MCAS  Yuma  and  with  all  the  aircraft  weight  and 
performance  information  preprogrammed.  Additionally,  a  WTI  control  point  database  was  loaded 
on  each  computer,  enabling  the  user  to  select  the  control  points  as  a  Local  Point  overiay  on  their 
FalconView  display. 

Because  of  the  core  PFPS  program's  relatively  small  computer  storage  requirement, 
approximately  233  Megabytes  (Mb),  students  and  instructors  were  able  to  load  copies  of  PFPS 
Version  3.0  on  their  Windows  95/NT  equipped  notebooks  or  desktop  PCs  for  additional  practice 
at  their  leisure. 

4.6  Assessment  Criteria.  The  Critical  Tactical  Issues  (CTIs)  that  were  addressed  during  this 
assessment  are;  a)  Training,  b)  Human  Factors,  c)  Documentation,  d)  Compatibility,  e)  Reliability, 
f)  Maintainability  and  g)  Tactical  Impact  of  PFPS.  These  CTIs  were  assessed  by  way  of  written 
surveys  that  the  respondents  filled  out.  Some  of  the  questions  are  quantitative  in  nature  and 
therefore  require  the  respondent  to  provide  a  number  or  text  input.  Other  questions  are 
qualitative  in  nature,  providing  a  Likert  rating  scale  for  the  respondent  to  express  their  level  of 
agreement  with  a  statement  about  the  suitability  or  effectiveness  of  PFPS.  Appendixes  D  is  the 
Initial  Survey,  designed  to  capture  background  information.  Appendix  E  is  the  Follow-on  Survey 
format  that  was  used  to  capture  the  opinions  of  the  respondents  on  the  CTIs. 


4 


Below  is  an  expansion  of  each  CTI,  highlighting  the  integral  measures  of  suitability  or 
effectiveness: 

4.6. a  CTI:  PFPS  Training 

Suitability  Test  (S-test):  Training 

Object:  Was  the  training  on  PFPS  during  WT1 1-98  adequate  enough  for  students 
to  mission  plan  with  PFPS? 

Procedure:  Training  issues  were  assessed  via  the  Training  section  of  the  survey. 

Measure  of  Suitability  (MOS):  The  Training  section  of  the  survey  addresses  the 
respondent's  opinion  on  the  adequacy  of:  a)  Formal  classroom  training,  b)  Formal 
hands-on  training,  c)  On-the-job  (OJT)  training  time  availability,  d)  hours  trained  on 
PFPS  and  e)  whether  the  respondent  feels  confident  enough  to  teach  PFPS  to 
their  squadron. 

4.6. b  CTI:  PFPS  Human  Factors 

S-test:  Human  Factors 

Object:  What  impact  did  PFPS  have  on  key  human-machine  interface  issues? 

Procedure:  Human  Factors  issues  were  assessed  via  the  Human  Factors  section 
of  the  survey. 

Measure  of  Suitability  (MOSV  The  Human  Factors  section  of  the  survey  addresses 
respondent's  opinion  on  the  level  of  difficulty  working  with  a)  Available  hardware,  b) 
PFPS  Graphical  User  Interface,  c)  PFPS  startup  sequence,  and  d)  Major  Falcon 
View  and  CFPS  menu  items. 

4.6. C  CTI:  PFPS  Documentation 

S-test:  Documentation 

Object:  Was  the  documentation  on  PFPS  adequate  enough  for  students  to 
mission  plan  with  PFPS? 

Procedure:  Documentation  issues  were  assessed  via  the  Documentation  section 
of  the  survey. 

Measure  of  Suitability  (MOS):  The  Documentation  section  of  the  survey  addresses 
respondent's  opinion  on:  a)  Hard  copy  user's  guides,  b)  On-line  help  and  c) 
Amplifying  instructions  in  pull-down/pop-up  menus. 

4.6. d  CI!i  PFPS  Compatibility 

S-test:  Compatibility 
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Object:  Were  the  modules  and  products  of  PFPS  compatible  with  the  available 
hardware  and  with  mission  planning  needs? 

Procedure:  Compatibility  was  assessed  via  the  Compatibility  section  of  the  survey. 

Measure  of  Suitability  (MOS):  The  Compatibility  section  of  the  survey  addresses 
the  respondent's  opinion  on  the  level  of  compatibility  between:  a)  PFPS  and  the 
Windows  95  operating  system,  b)  CFPS  and  FalconView,  c)  PFPS  print  options 
and  the  available  printer,  d)  PFPS  printed  products  with  mission  briefing/smartpack 
requirements,  and  e)  Major  PFPS  menu  selections  with  mission  planning 
requirements. 

4.6. e  CTT:  PFPS  Reliability 

S-test:  Reliability 

Object:  Was  PFPS  and  its  major  modules  reliable? 

Procedure:  Reliability  was  assessed  via  the  Reliability  section  of  the  survey. 

Measure  of  Suitability  (MOS):  The  Reliability  section  of  the  survey  addresses  the 
respondent’s  opinion  on  the  level  of  reliability  of:  a)  The  CFPS  module,  b)  The 
FalconView  module,  c)  The  Route  Server,  and  d)  PFPS  running  on  Windows  95. 

4.6. f  CTT:  PFPS  Maintainability 

S-test:  Maintainability 

Object:  Was  PFPS  maintainable  and  did  respondents  feel  knowledgeable  enough 
to  maintain  it  in  their  squadron? 

Procedure:  Maintainability  will  be  assessed  via  the  Maintainability  section  of  the 
survey. 

Measure  of  Suitability  (MOS):  The  Maintainability  section  of  the  survey  addresses 
the  respondent's  opinion  of  their  capability  to:  a)  Install  PFPS,  b)  Troubleshoot 
problems  with  PFPS,  c)  Access  technical  support. 

4.6. g  CTI:  PFPS  Tactical  Impact 

Effectiveness-test  (E-test):  Tactical  Impact 

Object:  What  would  be  the  impact  of  using  PFPS  to  plan  for  a  tactical  mission? 

Procedure:  Tactical  impact  was  assessed  via  a  single  question  in  the  General 
section  of  the  survey. 
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Measure  of  Effectiveness  (MOE):  The  questions  in  the  General  section  of  the 
survey  addresses  the  respondent's  opinion  of  the  level  of  significance  PFPS 
contributed  to  tactical  impact  in  planning  tactical  missions. 

For  application  of  advanced  statisics  under  the  assumptions  of  the  Central  Limit  Theorem, 
this  assessment  required  that  at  least  30  students  (not  previously  familiar  with  PFPS)  complete 
the  initial  PFPS  training,  continue  with  OJT  and  complete  all  3  survey  sessions. 

5.0  METHOD  OF  ASSESSMENT 

5.1  Assessment  Method  and  Procedures.  The  assessment  commenced  with  PFPS 
classroom  instruction  during  the  Common  Academics  Phase.  Upon  receipt  of  this  training,  the 
participants  were  surveyed  on  basic  demographics  and  questions  about  their  AMPS  and 
computer  experience.  After  this  initial  instruction,  students  used  after-class  hours  time  for  hands- 
on  with  one  of  the  three  computers  to  practice  working  with  PFPS.  Students  who  brought  their 
own  computers  had  PFPS  Version  3.0  installed  in  order  to  practice  mission  planning  in  a  self- 
paced  environment.  Evolution  Coordinators  ensured  as  much  as  possible  that  the  'PFPS  Guy' 
billet  rotated  as  the  flying  evolutions  progressed.  The  Baseline,  intermediate  and  Final  Surveys 
were  administered  after  the  Specific  Flying  phase,  after  Assault  Support  Tactics  and  at  the  end  of 
the  course  respectively  in  order  to  capture  trends  in  student  opinions.** 

5.1.  a  Pre-assessment  Checks.  Prior  to  the  start  of  WTI  1-98,  the  3-computer  suite  was  set  up 
in  the  mission  planning  room  and  PFPS  was  installed  on  each  computer.  Once  PFPS  was 
installed,  all  3  computers  had  the  WTI  1-98  control  point  data  base  installed  as  well  as  the 
baseline  aircraft-specific  routes  loaded  on  them. 

After  the  load,  all  facets  of  PFPS  applicable  to  the  assessment  were  run,  checked  for 
accuracy  (see  subsection  6.1  Operational  Risk  Management)  and  debugged.  To  facilitate 
implementing  software  improvements,  a  discrepancy  sheet  was  posted  by  each  machine  for 
users  to  document  problems  with  the  software.  Discrepancies  found  were  corrected  by  MAWTS- 
1  staff.  More  serious  problems  were  going  to  be  forwarded  expeditiously  to  the  software 
developer  as  that  information  is  outside  the  scope  of  this  assessment  and  need  not  be  held  until 
publishing  the  report  but  this  was  not  required. 

The  survey  designed  for  this  assessment  was  reviewed  for  validity  and  comprehensibility  of 
the  questions  prior  to  it  being  administered  to  the  students. 

5.1.  b  Pre-assessment  Training.  Initial  training  for  MAWTS-1  instructors  was  conducted  during 
July,  1997.  This  training  consisted  of  classroom  instruction,  a  demonstration  and  some  'hands- 
on'  instruction.  The  instructor  training  was  performed  by  Captain  Barr  and  Mr.  Jon  Drof  of  Litton 
PRC.  Litton  PRC  is  the  Navy's  contractor  for  conducting  fleet  training  for  the  Navy/Marine  PFPS 
program. 

During  WT1 1-98,  training  for  students  and  instructors  on  PFPS  consist  of  a  30  minute 
presentation  on  the  history  of  Navy  procured  AMPSs,  the  evolution  to  TAMPS  7.0  followed  by  a 
50  minute  'hands-on'  session  guided  by  an  instructor.  Table  1  contains  the  PFPS  Training 
Schedule  for  WT1 1-98.  After  the  initial  training  but  before  students  are  allowed  'hands-on' 


‘Completed’  surveys  did  not  necessarily  mean  that  all  questions  were  answered  -  the  results  reflect  this. 
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experience  with  PFPS,  all  students  completed  an  initial  survey.  Subsequent  training  as  stated 
was  on  a  self-paced  basis  by  the  student  or  as  the  mission  planning  billets  allowed. 


Table  1.  PFPS  Student  Training  Plan  for  WTt  1-98 


TRAINING  TYPE 

DIVISION 

DATE 

TIME 

INSTRUCTOR 

INITIAL 

KC-130 

18  SEP  97 

1030-1100 

CAPT  BARR 

ALL  ASD 

19  SEP  97 

1100-1130 

CAPT  BARR 

F/A-18 

***DND 

AV-8 

***DND 

EA-6 

***DND 

HANDS-ON 

KC-130 

18  SEP  97 

1030-1130 

CWO  GRIGALIS 

AH-1 

26  SEP  97 

1700-1750 

CAPT  KENNEDY 

UH-1 

29  SEP  97 

0700-0750 

CAPT  BARR 

CH-53 

29  SEP  97 

1410-1510 

CAPTWERNECKE 

CH-46 

29  SEP  97 

1510-1610 

CAPT  BARR 

AV-8 

***DND 

F/A-18 

***DND 

EA-6 

5.1. c  Operational  Procedures.  After  PFPS,  the  Control  Point  database  and  the  aircraft-specific 
files  were  installed,  the  computers  were  left  on  with  FalconView  and  CFPS  screens  active.  As 
planned,  if  the  computers  should  be  shut  down  or  if  either  the  FalconView  or  CFPS  modules 
should  crash,  restarting  PFPS  and  its  components  proved  to  be  extremely  easy.  From  the 
Windows  95  desktop,  PFPS  is  started  by  double-clicking  either  the  CFPS  or  FalconView  icon 
(Figure  5.1).  Once  either  CFPS  or  FalconView  is  running,  users  can  activate  the  other  module  by 
clicking  on  the  respective  icon. 


'  ;ir,.V-'Ty  .  • 


Figure  5.1.  Extract  from  the  Windows  95  desktop.  Depicted  on  the  desktop  are  the 
PFPS  and  FalconView  Icons,  as  well  as  the  Route  Server  icon  on  the  lower  bar.  Users 
simply  select  either  CFPS  or  Falconview  and  the  Route  Server  automatically  becomes 
active  to  coordinate  between  modules. 


*”  Tactical  Aviation  (TACAIR)  pilots  did  not  participate  in  this  QA  because  the  EA-6B  pilots 
mission  planned  with  TEAMS,  AV-8B  pilots  mission  planned  with  MOMS  (now  called  APS), 
and  FA-18  mission  planned  with  F-PLAN,  FOREM  and  TAMPS  (on  a  limited  basis). 
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Figure  5.2  is  a  view  of  the  Windows  95  desktop  with  both  CFPS  and  FalconView  windows 
open.  From  this  view,  CFPS  (left)  and  FalconView  (right)  as  well  as  their  respective  toolbars  can 
be  seen.  The  route  on  the  FalconView  map  display  is  what  is  being  automatically  updated  in  the 
CFPS  route  card  display.  It  is  also  important  to  note  the  taskbars  across  the  bottom  of  the 
display.  Although  the  Route  Server  is  not  visible  to  the  user,  the  presence  of  the  taskbar  shows 
that  it  is  active  and  is  coordinating  all  open  PFPS  modules. 


l^jTACtl  Combat  FJi^  Pl^riirtg  S pftwaie 
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Figure  5.2  PFPS  desktop  with  both  CFPS  (left)  and  FalconView  (right)  windows  active. 

The  route  on  the  FalconView  map  corresponds  to  the  waypoints  in  the  CFPS  display.  In  the 
bottom  of  the  figure,  the  Route  Server  taskbar  is  visible,  signifying  that  it  is  active  and 
coordinating  the  activity  of  the  CFPS,  PFPS  Point  Editor  and  FalconView  modules. 

5.2  Instrumentation  and  Data  Collection  &  Processing.  Data  was  collected  via  the  surveys  in 
Appendixes  B  and  C.  Appendix  B  is  the  initial  survey,  which  is  designed  to  gather  all  student 
demographics.  Appendix  C  is  designed  to  capture  opinions  on  Critical  Tactical  Issues  (CTIs). 
Table  2  is  the  survey  administration  plan  for  WT1 1-98. 


Table  2.  Survey  Administration  Plan  for  WT1 1-98 


The  MAWTS-1  Aviation  Operations  Analyst  constructed  a  database  of  all  survey  inputs. 
From  this  database  the  analyst  conducted  data  analysis  in  the  form  of  median  (middle),  mode 
(most  frequent),  maximum  and  minimum  scores  for  rating  scale  questions,  plots  of  responses 
versus  key  demographic  indicators,  plots  of  PFPS  utilization  and  trend  analysis  for  rating  scale 
questions  over  the  three-survey  period.  The  results  of  the  analysis  are  presented  in  section  8.0 
Results  And  Discussion. 

5.3  Support  Requirements.  The  following  support  requirements  were  required  to  execute  the 
assessment  during  WT1 1-98. 

5.3. a  MAWTS-1 

5.3. a.1  ADTSiE  Department 

a.  Coordinating  Officer/Aviation  Operations  Analyst; 

(1)  Co-developed  the  Qualitative  Assessment  plan. 

(2)  Co-developed  the  Initial  and  Follow-on  Surveys. 

(3)  Provided  analytical  assistance  to  support  assessment. 

(4)  Provided  assistance  to  Project  Action  Officer  in  writing  final  report. 

5.3. a.2  Assault  Support  Department 

a.  Project  Action  Officer: 

(1)  Provided  multiple  30  minute  introductory  briefings  on  PFPS  and  the  current  focus  of 
AMPSs  during  academics  for  WT1 1-98  students.  Coordinated  times/dates  with  division 
heads/PFPS  Model  Managers. 

(2)  Provided  multiple  50  minute  PFPS  hands-on  training  sessions  scheduled  during 
academics  for  WT1 1-98  students.  Coordinated  instructional  sessions  with  division  heads/PFPS 
Model  Managers. 

(3)  Co-developed  the  Qualitative  Assessment  plan. 

(4)  Co-developed  the  Initial  and  Follow-on  Surveys. 

(5)  Supervised  administering  and  collection  of  surveys. 

5.3.a.2  Participating  Departments 

a.  Participating  Division  Heads  or  Assigned  Division  PFPS  Model  Managers: 

(1)  Scheduled  30  minute  introductory  briefings  on  PFPS  and  the  current  focus  of  AMPSs 
during  academics  for  WT1 1-98  students.  Coordinated  times/dates  with  the  Project  Action  Officer. 
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(2)  Scheduled  50  minute  PFPS  hands-on  training  sessions  schedule  during  academics 
for  WT1 1-98  students.  Coordinated  instructional  sessions  with  the  Project  Action  Officer. 

(3)  Administered  and  collected  surveys  for  each  specific  divisions  and  returned  surveys 
to  the  Project  Action  Officer. 

5.3. b  PMA-233.  Provided  two  Pentium  laptop  computers  and  copies  of  PFPS  Version  3.0, 

Phase  2A  (beta). 

5.3, b  Litton  PRC.  Provided  initial  PFPS  instructor  training  at  MAWTS-1 . 

6.0  SAFETY  AND  SPECIAL  PROCEDURES 

6.1  Operational  Risk  Management  (ORM).  Appendix  D  contains  the  ORM  plan  for  this 
assessment. 

6.2  Environmental.  This  Qualitative  Assemsent  conducted  posed  no  adverse  threat  to  the 
environment.  No  significant  environmental  degradation  or  effect  is  known  to  be  occurring  as  a 
result  of  the  assessment  procedures;  therefore,  this  action  is  considered  not  significant  and 
requires  no  further  environmental  documentation. 

7.0  ADMINISTRATION  AND  LOGISTICS 

7.1  Manpower  Requirements.  Manpower  requirements  for  this  assessment  were  limited  to  the 
Project  Action  Officer,  the  Coordinating  Officer/Aviation  Operations  Analyst  and  the  PFPS  Model 
Managers  for  each  division  participating.  The  manpower  requirements  were  based  on  the  need 
to  conduct  pre-assessment  training  (see  paragraph  5.3)  and  to  administer/collect  the  surveys. 

7.2  Schedule/Milestones.  The  major  milestones  of  this  assessment  are  contained  in  Table  3. 


Table  3.  PFPS  Qualitative  Assessment  Milestones 


EVENT 

DATE(S) 

REMARKS 

PFPS  Training  by  Litton  PRC 

16  July  97 

Complete 

Arrival  of  PMA-233  Computer 
Equipment 

1  Sep  97 

Complete,  2  laptops 

Arrival  of  PFPS  Version  3.0 

1  Sep  97 

Complete 

Setup  of  3  Computer  Suite 

15  Oct  97 

In  work,  includes  control  point  &  aircraft 
databases 

PFPS  Student  Training 

18-29  Sep  97 

See  Table  1 

Surveys 

18  Sep  -  28  Oct  97 

After  initial  training,  Specific,  AST  and 
Finex  flight  phases 

Data  Analysis/Report  Preparation 

28  Oct  -  15  Dec  97 

Final  Report  Release 

20  Feb  98 

Requires  C.O.  MAWTS-1  Signature 

7.3  Personnel  Assignment.  Key  personnel  assignments  for  this  assessment  and  their 
information  is  listed  in  Table  4. 
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Table  4.  Personnel  Assignments  for  the  PFPS  Qualitative  Assessment 


RANK 

NAME 

BILLET 

E-MAIL 

FAX 

PHONE 

CAPT 

Barr,  R.  S. 

Project  Action  Officer/ 
CH-46  Model  MGR 

barrr@yuma.usmc.mil 

DSN  951-2637 

DSN  951-3469 

MAJ 

Sampson,  M.  T. 

Coordinating  Officer/ 
Operations  Analyst 

DSN  951-2637 

DSN  951-2498 

MAJ 

Bennett,  J.  S. 

PMA-233  Rep. 

DSN  757-8020 

DSN  757-7987 

CIV 

Medieros,  J. 

Msmsmmmfsm 

DSN  872-6379 

DSN  872-2350 

CAPT 

Wernecke,  S. 

CH-53  Model  MGR 

DSN  951-2637 

DSN  951-2643 

CAPT 

Kennedy,  B. 

AH-1  Model  MGR 

DSN  951-2637 

DSN  951-2967 

CAPT 

Vanderwerff,  J. 

UH-1  Model  MGR 

DSN  951-2637 

DSN  951-2132 

CWO 

Grigalis,  E.  A. 

KC-130  Model  MGR 

DSN  951-2637 

DSN  951-3570 

7.4  Reports.  This  assessment  will  be  complete  once  this  final  report  is  signed  by  the 
Commanding  Officer  and  the  distribution  list  has  received  their  copies.  Interim  reporting  will  be 
done  only  on  software  discrepancies  for  this  assessment.  In  the  instance  of  a  software 
discrepancy,  the  developer  will  be  contacted  as  soon  as  possible  to  ensure  the  final  version  of 
PFPS  3.0  is  debugged  as  much  as  possible  before  release  to  the  fleet.  Appendix  E  is  the 
Reports  Deliverables  Plan  that  was  signed  by  all  major  parties  involved  in  this  assessment. 

7.5  Project  Security.  This  assessment,  its  computer  hardware,  software  and  any  reports  are  or 
will  be  classified  'Unclassified'. 

8.0  RESULTS  AND  DISCUSSION  Overall  participation  in  the  assessment  by  the  supporting 
departments  was  good  with  roughly  40  respondents  contributing.  There  were,  however,  some 
lapses  in  participation  for  the  baseline  and  intermediate  survey  (Figure  8.1)  which  prompted 
‘rolling  up’  the  results  so  that  responses  to  each  questions  were  not  separated  for  trend  analysis 
across  the  assessment  period.  The  lack  of  at  least  30  complete  sets  of  surveys  prevented  higher 
level  analysis  under  the  assumptions  of  the  Central  Limit  Theorem,  however,  the  data  was  still 
acceptable  for  presentation  in  basic  statistical  terms.  Most  of  the  diagrams  in  this  section 
involving  ratings  provide  the  Median  (the  middle  value  of  all  values  placed  in  order)  and  the 
Mode  (most  frequent  value  chosen)  as  centrality  parameters,  and  the  Max  (maximum  value 
chosen)  and  the  Min  (minimum  value  chosen)  to  present  the  range  of  responses  given.  The 
rankings  section  below  also  includes  the  Mean  (average  of  all  values)  which  is  a  permissable 
centrality  parameter  in  that  case. 

PFPS  usage  prior  to  the  assessment  and  during  the  assessment  was  recorded  to 
provided  insight  into  the  PFPS  experience  level  of  tiie  respondents  at  the  outset  as  well  as  the 
amount  of  PFPS  usage  during  the  assessment  driving  the  responses  of  the  participants.  The 
respondents  were  regrouped  into  USMC  Rotary  Wing,  USMC  KC-130  and  USAF  Rotary  Wing  to 
better  highlight  the  different  experience  and  usage  levels.  Figure  8.2,  PFPS  Usage,  reflects  that 
those  most  familiar  with  PFPS  at  the  outset  (USAF  Rotary  Wing),  tended  to  use  PFPS  the  most 
during  the  assessment  and  that  average  usage  across  all  groups  remained  fairly  constant.  It  is 
important  to  note  that  KC-130  participation  in  the  Intermediate  Survey  was  not  available  and  was 
recorded  as  a  zero. 
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Survey  Participation  ^  Aircraft  Type 


AH-1W  CI-M6E  CI+53C)/E  KC-130  M+SSJ  UH-1N 

Aircraft  Type 


Figure  8.1.  Survey  Participation  by  Aircraft  Type.  Participation  is  depicted  by  aircraft 
type  and  then  broken  out  within  the  type  to  show  the  number  of  respondents  who 
participated  in  each  phase  of  the  survey.  The  most  respondents  participating  at  any  one 
time  was  41  and  the  least  was  23. 


PFPS  USAGE  REPORTED 


Previous 


HUSMCRW 
■  USAFRW 
□  USMC  KC-130 


10 

_J^ — j — HIHHHi - ^ — i 

Final 


Baseline  Intermediate 

Questionnaire  Period 


Figure  8.2.  PFPS  Usage  Reported  (Average  Hours).  Respondents  were  regrouped  into 
USMC  RW,  USMC  KC-130  and  USAF  RW  to  better  highlight  the  differences  in  usage  prior 
to  the  assessment  (‘Previous’)  and  up  to  the  Baseline,  Intermediate  and  Final  surveys.  The 
graph  reflects  that  those  most  familiar  with  PFPS  at  the  outset  (USAF  Rotary  Wing),  tended 
to  use  PFPS  the  most  during  the  assessment  and  that  average  usage  across  all  groups 
remained  fairly  constant  (Note:  USMC  KC-130  usage  between  the  Baseline  and 
Intermediate  survey  was  not  available  and  is  therefore  recorded  as  0). 
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8,1.  Background.  Basic  demographics  of  the  survey  respondents  were  recorded  in  the 
background  section  of  the  Initial  Survey.  Figure  8.3,  Respondent  Experience  Levels,  depicts  the 
Minimum,  Mean  (average)  and  Maximum  Values  for  Respondent  Age,  Years  of  Service,  Years  of 
Operational  Flying  and  Hundreds  of  Military  Flight  Hours.  In  the  opinion  of  the  Project  Action 
Officer,  the  respondents  in  this  survey  presented  a  good  crossection  of  the  senior  enlisted  to  field 
grade  military  aviator. 

In  Figure  8.4,  Computer  Experience,  Respondents  were  allowed  to  select  aj]  sources  of 
their  computer  experience.  The  results  reflect  that  most  had  PC  experience  (68%)  with  Network 
and  Macintosh  coming  in  a  distant  second  (15%)  and  third  (11%)  respectively.  None  of  the 
respondents  selected  Unix,  ‘Other*  or  ‘No’  computer  experience  as  their  top  3,  however,  in  later 
questions,  some  respondents  reported  having  worked  with  the  Unix  operating  system.  In  Figure 
8.4.,  Operating  System  Experience,  respondents  selected  all  types  of  operating  systems  they  had 
experience  with.  Most  had  Windows  95/97/NT  experience  (54%)  with  Windows  3.1  coming  in  a 
close  second  (30%)  and  Macintosh  coming  in  a  distant  third  (9%).  In  Figure  8.5,  Computer 
Training  History,  respondents  selected  where  they  received  most  of  their  computer  training. 
Almost  half  of  the  respondents  gained  most  of  their  experience  from  on-the-job  training  in  the 
Department  of  Defense  (48%).  Home  computing  was  a  close  second  (42%)  and  College  Degree 
Requirement  was  a  distant  third  (9%).  Few  (3%)  had  DOD  Formal  Training. 
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RESPONDENT  EXPERIENCE  LEVELS 


Figure  8.3.  Respondent  Experience  Levels.  Depicted  are  the  Minimum,  Mean  (average) 
and  Maximum  Vaiues  for  Respondent  Age,  Years  of  Service,  Years  of  Operational  Flying 
and  Hundreds  of  Military  Flight  Hours.  The  respondents  in  this  survey  presented  a  good 
crossection  of  the  senior  enlisted  to  field  grade  military  aviator. 


Figure  8.4.  Computer  Experience.  Respondents  selected  all  types  of  computers  they  had 
experience  with.  Most  had  PC  experience  (68%)  with  Network  and  Macintosh  coming  in  a 
distant  second  (15%)  and  third  (11%)  respectively.  None  of  the  respondents  selected  Unix, 
‘Other’  or  ‘No’  computer  experience  here. 
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OPERATING  SYSTEM  EXPERIENCE 


Figure  8.5.  Operating  System  Experience.  Respondents  selected  all  types  of  operating 
systems  they  had  experience  with.  Most  had  Windows  95/97/NT  experience  (54%)  with 
Windows  3.1  coming  in  a  close  second  (30%)  and  Macintosh  coming  in  a  distant  third  (9%) 
(Note:  The  Unix  operating  system  appears  here  when  it  was  not  indicated  in  Figure  8.3). 


Figure  8.6.  Computer  Training  History.  Respondents  selected  where  they  received  most 
of  their  computer  training.  Almost  half  of  the  respondents  gained  most  of  their  experience 
from  on-the-job  training  in  the  Department  of  Defense  (48%).  Home  computing  was  a  close 
second  (42%)  and  College  Degree  Requirement  was  a  distant  third  (9%).  Few  (3%)  had 
DOD  Formal  Training  and  none  had  prior  training  in  a  civilian  job. 
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In  Figure  8.7,  Previous  AMPS  Experience,  respondents  selected  all  sources  of  AMPS 
experience  they  had  prior  to  the  assessment.  Of  the  AMPS’s  available  in  the  DOD,  respondents 
had  most  experience  with  FPLAN  (30%).  TAMPS  ran  a  close  second  (25%)  and  PFPS  a  distant 
third  (11%).  Nine  percent  of  the  respondents  (approximately  4)  had  no  AMPS  experience.  Fifty 
percent  of  respondent  experience  was  with  FPLAN,  PFPS  and  FalconView  collectively.  These 
programs  are  PC-based  and  have  all  been  developed  by  the  46th  Test  Squadron,  based  at  Eglin 
AFB.  Figure  8.8  depicts  How  AMPS  experience  was  acquired.  Respondents  selected  whether 
their  AMPS  experience  was  acquired  under  one  or  a  combination  of  Training,  Exercise  or 
Operational  conditions.  Most  experience  was  acquired  under  training  conditions  (59%),  with 
exercise  conditions  second  (26%)  and  operation  conditions  a  distant  third  (15%).  In  the  opinion 
of  the  Project  Action  Officer,  the  percentages  realistically  reflect  an  average  DOD  Aviator’s  actual 
flying  experience  -  mostly  training,  a  lesser  amount  of  exercises  and  a  modest  amount  of 
operational  experience. 

In  Figure  8.9,  Squadron  Dedicated  AMPS  Computer,  respondents  indicated  what  was  the 
highest  level  of  computer  dedicated  to  mission  planning  in  their  squadron.  Most  (56%)  reported 
a  586  as  the  highest  and  486  (46%)  as  the  second  highest.  None  of  the  respondents  selected 
386  or  'No  dedicated  mission  planning  computer”.  In  the  opinion  of  the  Project  Action  Officer,  the 
respondents  misunderstood  the  question  because  numerous  reports  from  ‘Fleet  Support’  indicate 
there  was  no  computer  in  the  ready  room  and  computers  with  mission  planning  programs,  were 
‘multi-role’  or  not  dedicated  to  mission  planning.  The  Project  Action  Officer  interprets  the 
responses  to  indicate  the  highest  level  of  computer  in  the  unit,  indicating  that  units  are  able  to 
purchase  the  most  up-to-date  systems  -  whether  or  not  they  will  do  this  for  mission  planning, 
remains  to  be  seen. 


Previous  AMPS  Experience 

CHAMPS  none  other 


Figure  8.7.  Respondent  AMPS  Experience.  Of  the  AMPS’s  available  in  the  DOD, 
respondents  had  most  experience  with  FPLAN  (30%).  TAMPS  ran  a  close  second  (25%) 
and  PFPS  a  distant  third  (11%).  Nine  percent  of  the  respondents  (approximately  4)  had  no 
AMPS  experience.  Fifty  percent  of  respondent  experience  was  with  FPLAN,  PFPS  and 
FalconView  collectively.  These  programs  are  PC-based  and  have  all  been  developed  by 
the  46^*’  Test  Squadron,  based  at  Eglin  AFB. 
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HOW  AMPS  EXPERIENCE  WAS  ACQUIRED 


TRAINING 

59% 


Figure  8.8.  How  AMPS  Experience  Was  Acquired.  Respondents  selected  whether  their 
AMPS  experience  was  acquired  under  one  or  a  combination  of  Training,  Exercise  or 
Operational  conditions.  Most  experience  was  acquired  under  training  conditions  (59%),  with 
exercise  conditions  second  (26%)  and  operational  conditions  a  distant  third  (15%).  The 
percentages  realistically  reflect  an  average  DOD  aviator’s  actual  flying  experience  -  mostly 
training,  a  lesser  amount  of  exercises  and  a  modest  amount  of  operational  experience. 


SQUADRON  DEDICATED  AMPS  COMPUTER 


386  486 

0%  44% 


56% 


Figure  8.9.  Squadron  Dedicated  AMPS  Computer.  Respondents  indicated  what  was  the 
highest  level  of  computer  dedicated  to  mission  planning  in  their  squadron.  Most  (56%) 
reported  a  586  as  the  highest  and  486  (46%)  as  the  second  highest.  None  of  the 
respondents  selected  386  or  'No  dedicated  mission  planning  computer’.  Respondents  may 
have  mistakenly  reported  the  highest  level  of  computer  available  in  their  squadron. 
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8.2,  Training.  Below  is  an  excerpt  from  the  section  of  the  survey  on  training  (Median  scores 
boxed  in  red[j ).  Below  the  excerpt  is  Figure  8.10,  the  results  from  the  first  part  of  that  section, 
in  the  first  part,  respondents  were  asked  to  use  the  rating  scale  provided  to  rate  their  opinion  of 
the  level  of  adequacy  experienced  during  each  PFPS  training  evolution  by  circling  a 
corresponding  number. 

In  Figure  8.10,  Training  Ratings,  the  Mode  and  Median  for  each  question  reflect  that 
respondents  thought  the  initial  training  was  generally  adequate  but  that  they  were  neutral  on 
whether  or  not  the  OJT  or  ‘self-paced’  training  time  was  adequate.  The  Max  and  Min  for  each 
question  reflect  that  at  least  one  respondent  felt  all  the  training  was  very  adequate  or  better  while 
at  least  one  respondent  felt  the  training  was  inadequate  to  extremely  inadequate.  During  the 
Weapons  and  Tactics  Instructor  Course,  the  time  for  both  formal  and  ‘self-paced’  training  is 
limited  but  this  limit,  however  artificial,  is  similar  to  situations  pilots  may  face  during  fleet 
introduction  of  new  software  or  a  new  weapon  system  upgrade. 
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Figure  8.10.  Training  Ratings.  The  Mode  and  Median  for  each  question  refiect  that 
respondents  thought  the  initiai  training  was  generally  adequate  but  that  they  were  neutral  on 
whether  or  not  the  OJT  or  ‘self-paced’  training  time  was  adequate.  The  Max  and  Min  for 
each  question  reflect  that  at  least  one  respondent  felt  all  the  training  was  very  adequate  or 
better  while  at  least  one  respondent  felt  the  training  was  inadequate  to  extremely 
inadequate.  “Not  enough  training  time”  is  more  of  a  reality  than  an  artificiality. 
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In  the  Baseline  Survey  only,  respondents  were  asked: 


4)  With  your  training  in  PFPS  up  to  this  point,  would  you  be  comfortable  instructing  your  squadron  on  PFPS  use? 
(Circle  one)  Yes  No 

Figure  8.1 1  indicates  that  after  an  average  of  only  6.1  hours  of  training  time  since  the  start  of  the 
assessment,  64%  of  the  respondents  felt  that  they  would  feel  comfortable  instructing  their 
squadron  in  the  use  of  PFPS  while  36%  would  not.  Although  this  question  was  not  asked  on  the 
two  subsequent  surveys,  it  is  the  opinion  of  the  Project  Action  Officer  that,  based  on  the  results  of 
the  questions  on  training  above,  on  observations  and  on  interviews,  that  the  number  selecting 
‘Yes’  would  have  increased  into  the  percentile  or  higher  by  the  final  survey. 


BASELINE  SURVEY 
COMFORT  LEVEL  INSTRUCTING  PFPS 
(AVERAGE  6.1  HOURS  PRACTICE  SINCE  MTRODUCTION) 


Figure  8.11.  Comfort  Level  Instructing  PFPS.  Respondents  were  asked  to  answer  “Yes” 
or  “No”  if  they  would  feel  comfortable,  with  their  training  up  to  that  point  (the  Baseline 
Survey),  instructing  their  squadron  in  PFPS.  After  an  average  of  only  6.1  hours  of  training 
time  since  the  start  of  the  assessment,  64%  of  the  respondents  felt  that  they  would  feel 


comfortable  instructing  their  squadron  in  the  use  of  PFPS  while  36%  did  not. 


8.2.  Human  Factors.  Below  is  an  excerpt  from  the  section  of  the  survey  on  Human  Factors 
(Median  scores  boxed  in  red  □  )■  Below  the  excerpt  is  Figure  8.12,  the  results  from  that  section. 
Respondents  were  asked  to  rate  their  opinion  of  the  level  of  difficulty  experienced  while  using 
the  annotated  human-machine  interfaces  in  PFPS.  The  Median  and  Mode  for  all  questions  was 
five  or  better,  reflecting  that  respondents  found  that  all  human  machine  interfaces  were  ‘Easy’  to 
‘Very  Easy’  to  use.  Minimum  values  for  all  questions  reflected  that  none  of  the  areas  were 
‘trouble  free’.  Question  9,  editing  an  overlay,  received  the  lowest  median  (5)  and  the  lowest 
minimum  (1)  rating,  because  changing  aircraft  configuration  is  one  of  hardest  tasks  in  PFPS  due 
to  it  requiring  multiple  system  administrator-level  changes.  Only  C-130  and  F/A-18  flight 
performance  modules  were  available  at  the  time  of  the  assessment,  all  other  aircraft  required  the 
multiple  modifications. 
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1)  Working  with  available  PC/Laptop  hardware 
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3)  Working  with  the  PFPS  Graphical  User 
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interface  (GUI)  (i.e..  Desktop,  icons 
toolbars,  windows,  etc.). 

4)  PFPS  ‘boot-up’  sequence. 
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5)  Opening  a  CFPS  window. 
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6)  Opening  a  FalconView  window. 
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7)  Opening  a  saved  Overlay  File 
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(e.g.,  route,  threat,  waypoint,  etc.). 

8)  Saving  an  overlay  file. 
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9)  Editing  an  overlay  file  (e.g.,  change  route. 
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update  threat,  change  aircraft  configuration). 

10)  Printing  PFPS  products. 
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HUMAN  FACTORS  RESULTS 

LIKERT  RATING  SCALE 


Extremely  Hard  Neutral  Extremely  Easy 


Figure  8,12.  Human  Factors  results.  The  Median  and  Mode  for  all  questions  was  five  or 
better,  reflecting  that  respondents  found  that  all  human  machine  interfaces  were  ‘Easy’  to 
‘Very  Easy’  to  use.  Minimum  values  for  all  questions  reflected  that  none  of  the  areas  were 
‘trouble  free’  as  at  least  one  person  rated  each  interface  Neutral  or  less.  Changing  an 
overlay  (Question  9)  received  the  lowest  median  (5)  and  the  lowest  minimum  (1)  rating, 
possibly  reflecting  an  interface  requiring  improvement. 
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8.3  Documentation.  Below  is  an  excerpt  from  the  section  of  the  survey  on  documentation 
(Median  scores  boxed  in  red  □ ).  Below  the  excerpt  is  Figure  8.13,  the  results  from  that  section. 
Respondents  were  asked  to  rate  their  opinion  of  the  level  of  adequacy  experienced  while 
referring  to  the  following  documentation  sources  in  PFPS.  A  hard-copy  user’s  guide  was  not 
available  at  the  time  of  the  assessment,  accordingly  the  excerpt  and  Figure  8.13  have  been 
modified  to  reflect  this.  The  Median  and  Mode  for  Question  2  (online  help)  was  4,  reflecting  that 
respondents  found  the  documentation  was  niether  ‘Adequate’  nor  ‘Inadequate’.  The  Median  and 
Mode  for  Question  3  (pull-down/pop-up  menus)  was  5,  reflecting  that  respondents  found  them 
adequate.  Minimum  and  Maximum  values  for  all  questions  reflected  that  some  respondents 
found  the  documentation  ‘extremely  adequate’  while  others  found  it  to  be  ‘Inadequate’.  The 
ratings  indicate  that  documentation  needs  improvement. 
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3)  Descriptions/Instructions  accompanying 
a  pull-down/pop-up  menu  item  once  it  is 
selected  (e.g.,  description  box  for  map  scale 
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Figure  8.13.  Documentation  results.  The  Median  and  Mode  for  Question  2  was  4, 
reflecting  respondents  finding  the  documentation  niether  ‘Adequate’  nor  ‘Inadequate’.  The 
Median  and  Mode  for  Question  3  was  5,  reflecting  that  respondents  found  it  adequate. 
Minimum  and  Maximum  vaiues  for  all  questions  reflected  that  some  respondents  found  the 
documentation  ‘extremely  adequate’  whiie  others  found  it  to  be  ‘Inadequate’. 


8.4  Compatibility.  Below  is  an  excerpt  from  the  section  of  the  survey  on  Compatability  (Median 
scores  boxed  in  red  □ ).  Below  the  excerpt  is  Figure  8.14,  the  results  from  that  section. 
Respondents  were  asked  to  rate  their  opinion  of  the  level  of  compatibility  of  PFPS  and  the 
stated  requirement.  The  Median  and  Mode  for  all  questions  was  five  to  six,  reflecting  that  most 
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respondents  found  PFPS  ‘Compatible’  to  ‘Very  Compatible’  with  the  stated  functions.  Minimum 
and  maximum  values  for  all  questions  reflected  that  at  least  one  respondent  found  PFPS 
‘extremely  compatible’  while  at  least  one  found  certain  areas  to  be  ‘Incompatible’.  The  results 
indicate  that  PFPS  is  compatible  with  the  stated  funcitons. 
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3)  PFPS  print  options  with  available  printer. 
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4)  PFPS  printed  products  (i.e.,  route  cards, 
maps)  with  briefing/smartpack  requirements. 

5)  PFPS  Threat  Menu  selections  with 
mission  planning  requirements. 

6)  PFPS  Aircraft  Weight  &  Performance 
Menu  selections  with  mission  plaiming 
requirements. 

7)  PFPS  Route  Construction  menu  selections 
with  mission  plaiming  requirements. 

8)  PFPS  Airspace  Overlay  menu  selections 
with  mission  planning  requirements. 
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Figure  8.14.  Compatibility  results.  The  Median  and  Mode  for  all  questions  was  five  to 
six,  reflecting  that  most  espondents  found  PFPS  ‘Compatible’  to  ‘Very  Compatible’  with  the 
stated  functions.  Minimum  and  maximum  values  for  all  questions  reflected  that  at  least  one 
respondent  found  PFPS  ‘extremely  compatible’  while  at  least  one  found  certain  areas  to  be 
‘Incompatible’.  The  results  indicate  that  PFPS  is  compatible  with  the  stated  funcitons. 


23 


8.5  Reliability.  Below  are  two  excerpts  from  the  section  of  the  survey  on  Reliability  (Median 
scores  boxed  in  red  n ).  Below  the  first  excerpt  is  Figure  8.15,  the  results  from  the  first  section. 
In  the  first  section,  respondents  were  asked  to  rate  their  opinion  of  the  level  of  reliability 
experienced  while  using  PFPS.  Figure  8.15,  reflects  that  the  Median  and  Mode  for  all  questions 
was  6  which  means  respondents  found  the  stated  modules  very  reliable.  Min  values  ranged  from 
3,  unreliable  to  4,  neutral,  while  max  values  were  all  7’s,  extremely  reliable. 
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Figure  8.15.  Reliability.  The  Median  and  Mode  for  all  questions  was  6,  or  that 
respondents  found  the  stated  modules  very  reliable.  Min  values  ranged  from  in  unreliable 
to  neutral  while  max  values  were  all  extremely  reliable. 


In  the  second  part  of  the  Reliability  section,  a  more  quantitative  measurement  of  reliability 
was  measured  in  the  form  of  ‘crashes  per  session’  for  PFPS  and  the  Windows  95  operating 
system.  Although  this  assessment  was  not  focused  on  the  Windows  95  operating  system,  this 
measure  was  key  to  highlighting  the  reliability  of  PFPS  while  also  providing  an  alternate  measure 
of  its  compatibility  with  Windows  95. 

Below  is  the  excerpt  from  the  second  reliability  section  of  the  survey  and  below  that  are 
Figures  8.16  and  8.17,  the  results.  Figure  8.16  shows  that,  in  30  sessions,  users  experienced  no 
crashes  from  either  PFPS  or  the  Windows  95  operating  system  and,  on  another  13  sessions  (for 
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a  total  of  43  sessions),  Windows  95  had  no  crashes.  In  18  sessions,  PFPS  had  only  one  crash 
and,  in  4  other  sessions,  Windows  95  also  had  only  one.  Only  once  was  there  three  crashes  in  a 
session  with  PFPS  and  none  with  Windows  95.  Only  once  were  six  crashes  recorded  in  one 
session  for  each  (in  the  opinion  of  the  Project  Action  Officer,  this  report,  from  the  same 
respondent,  was  questionable).  Figure  8.17,  PFPS  Post-Crash  Results,  shows  that,  when  PFPS 
did  crash,  respondents  were  able  to  continue  20%  of  the  time  but  were  forced  to  end  the  session 
12%  of  the  time.  Another  12%  of  the  time,  respondents  had  mixed  results  and  either  were  able  to 
continue  or  the  session  ended  completely.  Again,  in  more  than  half  of  the  sessions,  respondents 
had  no  failures  from  PFPS.  In  light  of  these  results,  PFPS  is  determined  to  be  reliable. 


5)  While  using  PFPS,  approximately  how  many  times  per  session  did  PFPS  or  one  Of  its  modules  crash?  (stop 
working) 

6)  Did  these  crashes  end  your  session  or  were  you  able  to  restart  PFPS  or  the  failed  module(s)  and  continue? 
(Circle  one) 

Yes,  session  ended  No,  session  continued  Mixed  No  Crashes 

7)  While  you  were  using  PFPS,  approximately  how  many  times  per  session  did  Windows  95  crash?  (stop  working)  _ 
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RELIABILITY 
CRASHES  PER  SESSION 


NO  CRASHES  ONE  CRASH  TWO  CRASHES  THREE  CRASHES  FOUR  CRASHES  FIVE  CRASHES  SIX  CRASHES 


Figure  8.16.  Crashes  Per  Session  (PFPS  and  WIN  95).  The  results  show  that  in  30 
sessions,  users  experienced  no  crashes  from  either  PFPS  or  the  Windows  95  operating 
system.  In  18  of  the  sessions,  PFPS  had  only  one  crash  and,  in  4  of  the  sessions,  Win  95 
also  had  only  one.  Only  once  was  three  crashes  in  a  session  recorded  with  PFPS  and  none 
with  Windows  95.  Only  once  were  six  crashes  recorded  in  one  session  for  each  (suspect). 


RELIABILITY 

PFPS  POST-CRASH  RESULTS 


Figure  8.17.  PFPS  Post-Crash  Results.  In  the  times  when  PFPS  did  crash,  respondents 
were  able  to  continue  20%  of  the  time  but  were  forced  to  end  the  session  12%  of  the  time. 


Another  12%  of  the  time,  respondents  had  mixed  results  and  either  were  able  to  continue  or 
the  session  ended.  Again,  in  more  than  half  of  the  sessions,  respondents  had  no  failures. 
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8.6  Maintainability.  Below  is  an  excerpt  from  the  section  of  the  survey  on  Maintainability.  Below 
the  excerpt  is  Figure  8.18,  the  results  from  that  section.  Since  PFPS  will  eventually  be  introduced 
‘fleetwide’,  the  maintainability  questions  extended  beyond  the  assessment  to  project  the 
respondents  opinion  of  what  may  occur  during  the  introduction.  While  most  respondents  felt  that 
they  could  install  PFPS  and  access  technical  support,  few  felt  that  they  could  troubleshoot 
problems  with  PFPS.  However,  few  respondents  felt  that  PFPS  required  a  WTI  to  install  PFPS 
and  train  other  pilots  in  its  use.  Overall,  PFPS  is  determined  to  be  maintainable  at  the  squadron 
level  and  the  computer  background  of  the  respondents  here  is  seen  as  a  multiplier  -  like  the 
Windows/Windows  95  applications  most  are  used  to,  PFPS  requires  a  minimal  amount  of  special 
training  to  operate  and  maintain  it. 

1)  If  the  minimum  required  computer  (100  megahertz,  Pentium  PC,  running  Windows  95)  was  available  to  your  unit, 

would  you  be  comfortable  installing/reinstalling  PFPS  and  setting  up  the  required  databases  and  modules  for  your 
unit’s  use?  (Circle  one)  Yes  No  Don’t  Know 

2)  With  your  training  in  PFPS  up  to  this  point,  do  you  feel  comfortable  troubleshooting  PFPS  software  malfimctions? 

(Circle  one)  Yes  No  Don’t  Know 

3)  With  your  training  in  PFPS  up  to  this  point,  could  you  locate  information  on  how  to  access  technical  support? 

(Circle  one)  Yes  No 

4)  With  your  training  in  PFPS  up  to  this  point,  do  you  think  PFPS  requires  a  WTI  to  train  other  unit  personnel  in  its 

use?  (Circle  one)  Yes  No  Don’t  Know 


Figure  8.18.  Maintainability  Results.  While  most  respondents  felt  that  they  could  install 
PFPS  and  access  technical  support,  few  felt  that  they  could  troubleshoot  problems  with 
PFPS.  Also,  few  respondents  felt  that  PFPS  required  a  WTI  to  install  PFPS  and  train  other 
pilots  in  its  use.  Overall,  PFPS  is  determined  to  be  maintainable  at  the  squadron  level,  with 
the  maintainability  similar  to  that  of  other  Windows/Windows  95-based  applications. 
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8.7  General.  Below  are  four  excerpts  from  the  section  of  the  survey  covering  the  General 
Suitability  Issues  (Median  scores  boxed  in  red  Q  for  the  first  excerpt).  Below  the  first  excerpt  is 
Figure  8.19,  the  validity  rankings  of  each  of  seven  proposed  functionality  enhancements. 
Respondents  were  asked  to  rate  the  validity  of  each  functionality  enhancement  as  it  concerned 
their  community.  Median  and  Mode  ratings  for  all  Functionality  Enhancements  were  Neutral  to 
Valid  with  the  Takeoff  and  Landing  Data  (TOLD)  and  Hover  Mode  Tool  receiving  the  lowest  Mode 
ratings  (4).  The  low  rating  of  the  Hover  Mode  Tool  can  be  partially  attributed  to  the  KC-130 
participation  in  the  assessment  -  no  need  to  hover  there.  Mission  Data  Loader  (MDL)  received 
the  highest  Median  and  Mode  ratings  (7).  Maximum  values  reflect  that  one  or  more  respondents 
found  each  enhancement  valid  while  minimum  values  reflected  that, ,  unlike  the  other 
enhancements,  Map  Scan  and  Access  To  Other  Than  NIMA  Compressed  Graphics  were  never 
rated  Invalid. 


INVALID 


1)  Mission  Data  Load  (MDL/Brick)  capability.  1 

2)  Aircraft  Specific  Flight  Performance  Modules.  1 

(FPMs) 

3)  Takeoff  and  Landing  Data  (TOLD)  1 

weight  and  balance  calculation  capability. 

4)  Hover  Mode  Tool.  1 

5)  Circumlinear  (serpentine)  route  drawing  tool.  1 

6)  Capability  to  scan  in  any  map  and  use  it.  1 

7)  Capability  to  access  other  than  strictly  1 

NIMA-produced  Compressed  Arc  Digital 

Raster  Graphics(CADRG)  files. 
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FUNCTIONALITY  ENHANCEMENT  VALIDITY 

LIKERT  RATMG  SCALE 
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Figure  8.19.  Functionality  Enhancement  Validity.  Mode  and  Median  ratings  for  aii 
Functionality  Enhancements  was  neutrai  to  valid  with  the  Takeoff  and  Landing  Data  (TOLD) 
and  Hover  Mode  Tooi  receiving  the  lowest  Mode  ratings  (4).  The  low  rating  of  the  Hover 
Mode  Tool  can  be  partially  attributed  to  the  KC-130  participation  in  the  assessment  -  no 
need  to  hover  there.  Mission  Data  Loader  (MDL)  received  the  highest  Mode  and  Median 
ratings  (7).  Maximum  values  reflect  that  one  or  more  respondents  found  each 
enhancement  valid  whiie  minimum  vaiues  reflected  that  Map  Scan  and  Access  To  Other 
Than  NIMA  Map  Products  were  never  rated  invalid,  unlike  the  other  enhancements. 

In  Question  8,  the  second  part  of  the  General  section,  respondents  were  presented  with  a 
limited  funding  dilemma  and  asked  to  rank  the  mission  planning  system  functionality 
enhancement  listed  below  according  to  which  they  would  like  to  buy  for  their  community  -  first  all 
the  way  to  seventh,  using  the  numbers  one  through  seven  only  once.  Below  the  list  is  Figure 
8.20,  the  results  from  that  ranking,  arranged  left  to  right,  in  order  of  what  respondents  would  buy 
first  to  seventh.  Mean  (average)  rankings  for  all  functionality  enhancements  showed  that,  given  a 
limited  budget,  respondents  would  buy  an  MDL  for  their  aircraft  first,  the  Serpenting  Routing  Tool 
second,  Map  Scanning  Capability  third,  a  Flight  Performance  Module  for  their  aircraft  fourth, 
access  to  other  than  NIMA  Compressed  Graphics  fifth,  a  TOLD  module  sixth  and  a  Hover  Mode 
Tool  seventh.  The  Modes  for  each  enhancement  conformed  to  this  hierarchy  except  that  FPM 
and  Non-NIMA  tied  for  fourth. 

_ Mission  Data  Load  (MDL/Brick)  capability 

_ Aircraft  Specific  Flight  Performance  Modules  (FPMs) 

_ Takeoff  and  Landing  Data  (TOLD)  weight  and  balance  calculation  capability 

_ Hover  Mode  Tool 

_ Circumlinear  (serpentine)  route  drawing  tool 
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Capability  to  scan  in  any  map  and  use  it 

Capability  to  access  other  than  strictly  NIMA-produced  compressed  digital  graphics 


FUNCTIONALITY  ENHANCEMENT  RANKINGS  (ALL  COMMUNITIES) 
IN  ORDER  OF  DESIRE,  FIRST  TO  SEVENTH 


Figure  8.20.  Functionality  Enhancement  Rankings.  Mean  (average)  rankings  for  all 
Functionality  enhancements  showed  that,  given  a  limited  budget,  respondents  would  buy  an 
MDL  for  their  aircraft  first,  the  Serpenting  Routing  Tool  second.  Map  Scanning  capability 
third,  a  Flight  Performance  Module  for  their  aircraft  fourth,  access  to  other  than  NIMA 
compressed  graphics  fifth,  a  TOLD  module  sixth  and  a  Hover  Mode  Tool  seventh.  The 
Modes  for  each  enhancement  conformed  to  this  hierarchy  except  that  FPM  and  Non-NIMA 
tied  for  fourth. 

In  Question  9  of  the  General  section,  respondents  were  given  the  opportunity  to  write  down 
other  functionality  enhancements,  not  listed  in  Question  8,  that  they  would  like  to  see  in  future 
AMPS’s.  Their  responses  are  listed  below; 

-  “The  ability  to  change  route  card  setup.” 

-  “3-D  View  of  route.” 

-  ’’Incorporate  lite  level  planning,  calculate  shadows  based  on  moon  position.” 

-  “Smartpackage  creator  menu.” 

-  “The  ability  to  change  aircraft  models.” 

In  Question  10  and  11  of  the  General  section,  respondents  were  asked  to  rate  the  level  of 
Tactical  Impact  and  Time  Savings  respectively  that  PFPS  would  have  on  missions  in  their 
particular  community.  Below  Is  the  exc^erpt  from  the  survey  employed  (Median  scores  boxed  in 
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reclQ).  Below  the  excerpt  is  Figure  8.21,  the  results  from  that  rating.  Respondents  rated  PFPS 
high  on  Tactical  Impact  as  the  Mode  and  Median  ratings  were  significant  to  very  significant.  At 
least  one  respondent  was  neutral  on  the  Tactical  Impact  of  PFPS:  Time  Savings  using  PFPS  was 
rated  even  higher  with  the  Mode.  Max  and  Median  all  showing  Time  Savings  as  very  significant. 

At  least  one  respondent  felt  the  Time  Savings  was  somewhat  significant. 


10)  Using  the  scale  below,  rate  the  level  of  Tactical  Impact  using  PFPS  would  provide  have  on  missions  in  your  community. 


VERY 

NEUTRAL 

VERY 

mSIGNmCANT 

SIGNIFICANl 

Tactical  Impact. 

1  2 

3  4  5 

□  7 

1 1)  Using  the  scale  below,  rate  the  level  of  Time  Savings  using  PFPS  would  provide  have  on  missions  in  your  community. 


Time  Savings. 


2  3  4  5  6  □ 


Figure  8.21.  Tactical  Impact  and  Time  Savings.  Respondents  rated  PFPS  high  on 
Tactical  Impact  as  the  Mode  and  Median  ratings  were  significant  to  very  significant.  At  least 
one  respondent  was  neutral  on  the  Tactical  Impact  of  PFPS.  Time  Savings  using  PFPS 
was  rated  even  higher  with  the  Mode,  Max  and  Median  all  showing  Time  Savings  as  very 
significant.  At  least  one  respondent  felt  the  Time  Savings  was  somewhat  significant. 
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9-0  CONCLUSIONS  The  assessment  results  and  conversations  with  the  respondents 
highlighted  the  fact  that  the  majority  of  the  fleet  still  uses  a  'stubby  pencil’  and  CR2  slide 
rules  or  a  combination  of  existing  AMPSs  in  different  work-around  modes  to  complete 
simple  mission  planning  tasks.  In  other  words,  no  single  AMPS  has  been  developed 
which  has  been  adopted  by  DoN  aviators  or  which  meets  all  their  automated  mission 
planning  needs. 

The  conclusions  provided  below  are  based  on  the  assessment  results,  augmenteds  with  the 
professional  opinion  and  observations  of  the  Project  Action  Officer.  This  section  has  been 
organized  into  Enhancing  Characteristics  and  Deficiencies  and  subdivided  into  the  applicable 
Critical  Tactical  Issues  explored  in  the  assessment. 

9.1  Enhancing  Characteristics 

9.1. a  Training.  Training  consisted  of  a  30  minute  introduction  to  PFPS,  a  50  minute  hands-on 
presentation  (computer  lab)  and  on-the-job  training  via  billet  requirements  or  in  a  self-paced 
environment.  In  light  of  the  relatively  steady  average  usage  of  PFPS  during  the  course  (Figure 
8.2),  and  the  adequacy  ratings  for  the  training  (Figure  8.10),  training  with  PFPS  was  simple 
enough  for  almost  two  thirds  of  the  respondents  to  say  they  felt  comfortable  instructing  PFPS  to 
their  squadron  after  only  an  average  of  6.1  hours  of  training  time  (Figure  8.1 1).  The  majority  of 
computer/operating  system  experience  of  the  respondents  being  PC/Windows-based  (Figures  8.4 
and  8.5)  is  'fleet  representative’  and  no  doubt  aided  in  the  rapid  assimilation  of  PFPS. 

9.1. b  Human  Factors.  The  Human  Factors  section  covered  all  basic  hardware/software 
interfaces  of  PFPS.  All  aspects  of  the  PFPS  human-machine  interface  appeared  ‘very  easy’  to 
‘extremely  easy’  to  use.  Again,  the  PCAAfindows/Windows  95  background  of  a  majority  of  the 
respondents  was  believed  to  be  a  strong  influence  here. 

9.1.  c  Documentation.  None  noted. 

9.1. d  Compatibility.  The  Compatibility  section  covered  the  compatibility  of  PFPS  with  the 
operating  system,  within  its  own  modules  and  with  the  mission  planning  needs  of  the  users. 

PFPS  was  found  to  be  ‘compatible’  to  ‘very  compatible’  with  hardware/soflware  and  user  needs 
(Figure  8.14). 

9.1.  e  Reliability.  The  Reliability  section  covered  the  reliability  of  the  Route  Server  and  sub- 
modules  of  PFPS  as  well  as  the  reliability  of  PFPS  running  on  Windows  95.  Crashes  per  session 
for  both  PFPS  and  Windows  95  were  recorded  as  well  as  post-crash  results  for  PFPS.  PFPS 
was  very  reliable,  having  relatively  few  crashes  and  those  crashes  rarely  resulting  in  the  end  of  a 
session  (reboot  of  PFPS  required)  (Figures  8.15,  8.16,  8.17). 

9.1.  f  Maintainability.  The  Maintainability  section  covered  the  user’s  opinion  of  whether  or  not 
they  would  be  able  to  install  PFPS  and  set  up  the  required  databases,  troubleshoot  PFPS 
software  malfunctions,  access  technical  support  and  whether  or  not  PFPS  requires  a  Weapons 
and  Tactics  Instructor  to  train  unit  personnel  in  its  use.  PFPS  is  easy  to  maintain  due  to  ease  of 
installation,  ease  of  accessing  technical  support  and  that  a  WTI  or  ‘specialist’  is  not  required  to 
train  others  in  its  use  (Figure  8.18). 
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9.1.  g  Tactical  Impact/Time  Savings.  In  the  Tactical  Impact  and  Time  Savings  subsections, 
respondents  were  asked  to  simply  rate  the  significance  of  the  tactical  impact  and  time  savings 
PFPS  would  have  on  a  mission  in  their  particular  community.  PFPS  would  have  a  very  significant 
tactical  impact  simply  because  of  time  savings  in  making  changes  in  an  environment  where 
change  is  constant  (Figure  8.18).  Time  savings  and  tactical  impact  will  be  further  boosted  with 
the  advent  of  the  hardware  to  support  Mission  Data  Loaders. 

9.2  Deficiencies 

9.2. a  Training.  The  absence  of  an  ‘on-line’  tutorial  that  would  provide  the  user  ‘How  to . 

necessitated  the  formal  and  laboratory  training  given  prior  to  the  assessment.  A  tutorial,  such  as 
a  ‘Coach’  or  help  menu  selection  consistent  with  user  background  (PC/Windows-based)  may 
eventually  eliminate  the  need  for  this. 

9.2. b  Human  Factors.  Question  9  of  the  Human  Factors  section  asked  the  respondents  about 
the  level  of  difficulty  experienced  while  editing  an  overlay  file  (change  route,  update  threat, 
change  aircraft  configuration).  Changing  aircraft  configuration  was  the  sole  reason  for  the  lower 
median  rating  in  this  question  while  attempting  wholesale  changes  in  the  system  admirijstration 
limits  and  aircraft  configuration  files.  This  process  requires  improvement  to  make  it  more  simple 
and  make  it  more  intuitive  for  the  user. 

9.2. C  Documentation.  Respondents  were  at  best  neutral  about  the  adequacy  of  the  online  help 
because  it  is  limited.  Documentation  could  be  improved  to  offer  more  help  menu  selections  of  the 
most  common  tasks  users  experience  difficulty  with. 

9.2. d  Compatibility.  None  noted. 

9.2. e  Reliability.  None  noted. 

9.2. f  Maintainability.  Related  to  the  Training  and  Documentation  deficiencies  noted  above  was 
the  lack  of  tutorials  or  instructions  to  assist  users  installing  PFPS  and  troubleshooting  problems 
with  the  modules.  This  problem  has  partially  been  corrected  with  the  publishing  of  the  hard  copy 
Installation  Guide,  however,  an  ‘Install  Wizard’  and  online  troubleshooting  consistent  with  user 
computer  experience  would  be  more  practical. 

9.2. g  Tactical  Impact/Time  Savings.  None  noted. 

9.2. h  Other.  Currently,  aircraft-specific  overlays  for  a  particular  mission  (i.e.,  routes,  threats, 
drawings)  must  be  saved  in  separate  files  and  opened  separately.  The  capability  should  exist  to 
create  a  mission  folder  that  would  contain  all  aircraft  routes  for  a  mission  as  well  as  threats  and 
mission-specific  drawings  (phaselines,  etc.). 
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10.0  RECOMMENDATIONS  The  recommendations  provided  below  are  based  on  the 
assessment  results  and  the  conclusions  in  the  previous  section  and  they  are  augmented  with  the 
professional  opinion  and  observations  of  the  Project  Action  Officer.  This  section  has  been 
organized  into  a  General  section,  providing  ‘big  picture’  recommendations  and  a  Specific  section, 
recommending  more  detailed  actions. 

10.1  General  nurino  the  assessment.  PFPS  was  very  well  received  as  an  extremely  valuable 
Planning  tool  and  it  was  quickly  assimilated  bv  the  respondents.  For  basic  mission  planning 
functionality  as  well  as  for  preparing  reuseable  briefing  products  (electronic  and  hard  copies),  its 
speed  and  ease  of  use  was  unmatched  bv  any  single  AMPS.  Accordingly,  the  following  general 
recommendations  to  CNO  N62,  CNO  N880  and  PMA-233  are  provided: 

10.1.  a  Provide  rapid  dissemination  of  the  most  current  version  of  PFPS  to  all  DoN  squadrons  as 
soon  as  possible. 

10.1.  b  Plan  and  provide  for  rapid  implementation  of  the  top  three  (or  more)  functionality 
enhancements  (resources  permitting)  identified  in  this  report. 

10.1. C  Augment  existing  fleet  AMPS  computer  hardware  with  limited  laptop  and  desktop 
computer  buys  -  this  must  include  Mission  Data  Loader  Receptacle  (MDLR)  procurement. 

10.1.  d  Fund  rapid  correction  of  all  deficiencies  with  PFPS  noted  in  this  assessment  report. 

10.1  .e  Cosidering  the  computer  experience  of  the  assessment  respondents  and  the  fleet  and 
considering  the  results  of  this  assessment,  recommend  any  follow-on  AMPS  procured 
(JMPS/TAMPS  7.0)  conform  with  IT-21  standards  and  be  modeled  after  PFPS.  PC-based, 
Windows  95/NT  compatible,  point-and-click,  drag-and-drop,  etc.  are  some  of  the  qualities  that 
future  AMPS’s  should  possess. 

10.2  Specific 

10.2. a  In  light  of  the  assessment  respondent’s  computer  training  history  (home  PC  and  DOD 
OJT)  and  software  developers  and  academic  institutions  moving  toward  Computer  Based 
Training  (CBT),  recommend  steering  improvements  in  PFPS  Training  toward  an  ‘on-demand’ 
online  tutorial.  This  capability  should  include  an  'Install  Wizard’,  introductory  training  session  that 
provides  live  results  and  a  ‘Coach’  function  available  at  any  time  for  most  of  the  more  common 
tasks. 

10.2. b  Until  the  capability  above  is  available,  recommend  introductory  training  (similar  to  what 
was  done  for  this  assessment)  be  provided  that  will  enable  fleet  acceptance  of  PFPS  and  aid  in 
the  preparation  for  introduction  of  TAMPS  7.X. 

10.2. C  Fund  the  fielding  of  a  hard  copy  user’s  guide  to  augment  the  online  help  and  satisfy  the 
needs  of  users  who  are  not  familiar  with  using  online  help. 

10.2. d  Fund  PFPS  Mission  Data  Loader  Receptacle  (MDLR)  hardware  and  software  upgrades 
for  all  DON  squadrons. 

10.2. e  Fund  PFPS  Serpentine  (Circumlinear)  route  drawing  tool  capability. 
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10.2. f  Fund  PFPS  Map  Scan  capability.  This  capability  should  include  the  ability  to  scan  in  any 
map  (overhead  imagery,  etc.)  and  georectify  it  by  identifying  (giving  lat-long,  grid,  etc.)  at  least  3 
known  points  (triangulation). 

10.2. g  Fund  Hover  Mode  Tool  capability.  Although  DoN  Tactical  Aviation  and  C-130s  would 
have  no  use  for  this  tool,  the  high  percentage  of  Marine  Corps  rotary  wing  (and  eventual  tilt  rotor) 
aircraft  makes  this  tool  a  must.  All  DoN  rotary  wing  aircraft  will  benefit  from  this  enhancement. 

10.2. h  Fund  the  remaining  Flight  Performance  Modules  for  all  aircraft  in  the  Naval  Aviation 
Inventory  and  any  aircraft  currently  under  development  (i.e.,  V-22). 

10.2.i  Fund  the  capability  to  create  a  single  mission  folder  to  hold  multiple  route,  threat  and 
drawing  overlays.  This  capability  must  include  the  ability  to  ‘clutter*  and  ‘declutter*  the  display  as 
required  for  a  mission  briefing/smartpack. 
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CFPS  -  Combat  Fliaht  Plannina  Software 


■  Easy  editing  with  “Form  70"-style  Tabular  view 

■  Fully  supports  Search  and  Retrieval  from  a  standard 
DAFIF  based  library  of  Navaids  (VOR,  TACAN), 
Waypoints  (IFR  and  Low  Level)  and  User-Defined 
Points 

■  Inflight  Planning  from  Departure  to  Landing  with 
Customized  Knee-Board  style  Cards 

■  Complete  Takeoff  and  Landing  Data  Card  Preparation 

■  Fully  Supported  with  DoD  Certified,  Technical  Order 
Based,  Flight  Performance  Data 

■  Interoperable  with  other  mission  planning  systems 
via  data  file  exchange 


FalconView  -  Digital  MaPDina  Software 

■  Fully  Integrated  with  CFPS 

■  "Point  and  Click"  Route  Editing 

■  DMA  Standard  World-Wide  Maps,  DTED,  and 
Photo-Imagery 

■  Moving  Map  with  GPS  for  Inflight  Positioning 
Capability 
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Engagement  Capabilities 
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«  Supports  a  wide  range  of  munitions  -  General 
Purpose  Bombs  through  CBUs,  including 
CEM,  Sensor  Fuzed  Weapon,  and  most  Laser 
Guided  Bombs. 

■  Incorporates  USAF  certified  Safe  Escape 
and  Safe  Separation  methodology 

■  Provides  USAF  standard  Ballistic  and  Safe 
Escape  tabular  data 


CAPS-Combat  AirProp  Planning  Software 

•  Fully  Integrated  with  CFPS  and  FalconView 

■  Complete  Computed  Air  Release  Point  (CARP) 
and  High  Altitude  Release  Point  (HARP)  Solutions 

■  Certified  Drop  Zone  (DZ)  Database 

■  DZ  Imagery 

•  Automated  DZ  Survey  Support 

>ported  Airframes;  C-1 30,  C-1 41 ,  C-1 7, 

■  Supports  all  USAF  Airdrop  communities:  AMC, 
ACC,  AFSOC,  and  ANG 
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CTPS  -  Combat  Taraet  Plannina  Software 


Provides  JMEM  Probability  of  Damage  (l^)for  a 
range  of  delivery  parameters 

PoOutputs  prioritized  highest  to  lowest  for  selected 
aircraft  and  munition  against  a  JMEM  target 

Supports  a  wide  range  of  munitions  and  fighter 
aircraft 


Cartridge  Load 

■  Fully  Integrated  with  CFPS 

■  Supports  the  Full  Range  of  Cartridge  Needs;  Route, 
Communication,  Navigation,  Weapons,  etc. 

•  Currently  Supported  Airframes;  F-1 6s,  B-52,  B-l ,  F-ll  7 
Interface 

■  Future  Support;  C-1 30  Family,  HH-60 

■  Supports  Standard  Mission  Download  for  Debriefing 
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Ask  Your  Technical  Order  Distribution  Officer 


46TS/OGEr 

205  WDAve.  Suite  628 

EglinAFB,FL  32542-6865 

Voice:  (904)  882-3527,  DSN  872-3527 

FAX:  (904)  882-6379,  DSN  872-6379 
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Call  the  System  Support  Facility  (SSF)  for  PFPS 

1-800.SSF-SSFX 

Recommended  Hardware: 

Pentium™-class  PC 

16MB  ofRAM,  1GB  Hard  Disk 

4X  CD-ROM  Drive 

17"  monitor  (desktop)  or  800  x  600 
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APPENDIX  B.  INITIAL  SURVEY 


PORTABLE  FLIGHT  PLANNING  SOFTWARE  (PFPS) 

INITIAL  QUESTIONNAIRE 

Instructions.  This  questionnaire  is  designed  to  collect  information  on  your  computer  experience  and  mission 
planning  experience  up  to  this  point.  In  later  surveys  you  will  be  given  an  opportunity  to  give  your  opinion  about 
mission  planning  system  needs  for  yom  specific  community.  Your  participation  in  the  follow-on  surveys  is  critical 
to  getting  the  best  mission  planning  system  out  to  the  fleet  Please  participate! 

Background  Information. 

Name:  _  Rank  (i.e.,  03):  _  Service  (i.e.,  USMC):  _ 

Age:  _ 

MOS  Description  (circle  or  write  in  one):  CH-46E  CH-53D/E/J  AH-IW  UH-IN  KC-130 

F/A-18  AV-8 

EA-6B  Intelligence  GCE  Other:  . _ 

Years  of  Service: _ Years  of  Operational  Flying: _  Total  Military  Flight  hours: _ 

Computer  Experience  (circle  or  write  in  one  or  more): 

PC  MAC  Network  Mainframe  None  Other:  _ 

If  other  than  None  above,  what  is  your  operating  system  experience  (circle  or  write  in  one  or  more): 

DOS  MAC  OS2  Unix  Win95/97/NT  None  Other:  _ 

If  other  than  None  above,  what  is  your  program  interface  (i.e.,  Graphcal  User  Iner&ce)  experience: 

Win  3.1  MAC  OS2  Unix  Win95/97/NT  None  Other:  _ 

What  is  the  highest  level  computer  capability  available  in  yom  unit  available  for  (or  dedicated  to)  mission  planning? 
(Circle  or  write  in  one) 

386  486  586  (Pentium)  None  Other:  _ 

If  you  have  computer  experience,  what  do  you  attribute  most  of  your  computer  experience  to?  (Circle  or  write  in  one) 
DOD  On  the  Job  Training  Home  Computing  Formal  DOD  Computer  Training 

College  Degree  Requirement  Civilian  Job  Training  Other:  _ 

Mission  Planning  System  Experience  (circle  or  write  in  one  or  more): 

PFPS  TAMPS  TOPSCENE  FalconView  FPLAN  CHAMPS  FOREM  None  Other:  _ 

If  other  than  None  above,  was  your  experience  with  Mission  Planning  Systems  under;  Training  Exercise 
Operation  conditions  (circle  one  or  more). 

Approximate  hours  of  previous  ‘hands-on’  experience  using  PFPS:  _ 


APPENDIX  C.  FOLLOW-ON  SURVEY 


PORTABLE  FLIGHT  PLANNING  SOFTWARE  (PFPS) 
BASELINE  QUESTIONNAIRE 


Instructions.  This  questic»inaire  is  designed  to  collect  information  on  your  experiences  using  PFPS.  This  is 
also  an  opportunity  for  you  to  give  your  opinion  about  tnissicm  plannjng  system  needs  for  your  specific 
community.  The  survey  ccmsists  of  a  background  informaticai  secticm  followed  by  seven  sections  of  questions 
on;  (1)  Training,  (2)  Human  Factors,  (3)  Documentation,  (4)  Compatibility,  (5)  Reliability,  (6) 
Maintainability,  and  (7)  General.  Comments  can  be  made  after  each  questicm.  If  you  have  not  already 
completed  the  hiitial  Questionnaire,  asking  you  for  background  information,  please  ask  your  division  PFPS 
Model  Manager  to  get  you  one. 

Your  participation  in  this  and  subsequent  survey  periods  is  critical  to  getting  the  best  mission  planning  system 
out  to  the  fleet  Please  continue  participating! 

Background  Information. 

Name:  _  Rank(i.e.,  03):  _  Service  (i.e.,  USMC);  _ 

I  have  completed  the  Initial  Survey  asking  me  all  pertinent  background  information  (circle  one).  Yes  No 


L  TRAINING 

Using  the  scale  below,  rate  your  opinion  of  the  level  of  adequacy  experienced  during  each  PFPS  training  evolution  by 
circling  a  corresponding  number.  Comments  may  be  made  after  each  question. 


EXTREMELY 

INADEQUATE 

1)  The  30  minute  Introduction  to  PFPS.  1 

Comments/What  could  be  better?: 

2 

3 

NEUTRAL 

4 

5 

6 

EXTREMELY 

ADEQUATE 

7 

2)  The  50  minute  ‘Hands-On’  Presentation.  1 

Comments/What  could  be  better?: 

2 

3 

4 

5 

6 

7 

3)  On-the-Job  (OJT)  training  time  availability.  1 

Comments/What  could  be  better?: 

2 

3 

4 

5 

6 

7 

4)  With  your  training  in  PFPS  up  to  this  point,  would  you  be  comfortable  instructing  your  squadron  on  PFPS  use? 

(Circle  one)  Yes  No 

5)  ^)proximate^  how  many  hours  of  ‘hands-on’  PFPS  experience  have  you  had  once  last  survey? _ 

n.  HUMAN  FACTORS 


Using  the  scale  below,  rate  your  opinion  of  the  level  of  difficulty  experienced  while  using  the  following  human- 
machine  interfaces  in  PFPS  by  cirdling  a  corresponding  number.  Comments  may  be  made  after  each  question. 


EXTREMELY 

NEUTRAL 

EXTREMELY 

HARD 

EASY 

1)  Working  with  available  PC/Laptop  hardware  1  2 

(e.g.,  mouse,  keyboard,  touchpad). 

3  4  5 

6  7 

Comments/What  could  be  better?: 


EXTREMELY 

HARD 


NEUTRAL 


EXTREMELY 

EASY- 


2)  Working  with  available  printer  hardware 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

3)  Working  with  the  PFPS  Graphical  User 
interface  (GUI)  (i.e.,  Desktop,  icons 
toolbars,  windows,  etc.)i 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

4)  PFPS  ‘boot-up’  sequence. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

5)  Opening  a  CFPS  window. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

6)  Opening  a  Falcon  View  window. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

7)  Opening  a  saved  Overlay  File 
(e.g.,  route,  threat,  waypoint,  etc.). 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

8)  Saving  an  overlay  file. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

9)  Editing  an  overlay  file  (e.g.,  change  route, 
update  threat,  change  aircraft  configuration). 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

10)  Printing  PFPS  products. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

HL  DOCUMENTATION 

Using  the  scale  below,  rate  your  opinion  of  the  level  of  adequacy  experienced  while  referring  to  the  following 
documentation  sources  in  PFPS  \sy  circling  a  corresponding  number.  Comments  may  be  made  after  each  question. 


EXTREMELY 

INADEQUATE 


NEUTRAL 


EXTREMELY 

ADEQUATE 


1)  Hard  copy  FFPS  user’s  guide. 

Comments/What  could  be  better?: _ 

2)  PFPS  online  ‘he4)’. 

Comments/What  could  be  better?: _ 

3)  Descriptions/Instructions  accompanying 
a  pull-down/pop-up  menu  item  once  it  is 
selected  (e.g.,  description  box  for  map  scale 
choices,  etc.). 


Comments/What  could  be  better?: 


IV.  COMPATTBELITY. 


Using  the  scale  below,  rate  your  opinion  of  the  level  of  compatibility  experienced  between  PFPS  and  the  stated 


software/mission  planning  requirement  by  circling  a  corresponding 
question. 

EXTREMELY 

INCOMPATIBLE 

1)  PFPS  with  the  Windows  95  operating  system.  1  2 

Comments/What  could  be  better?: 

number. 

3 

Comments  may  be  made  after  each 

NEUTRAL  EXTREMELY 

COMPATIBU 

4  5  6  7 

2)  CFPS  module  with  FalconView  module. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

7 

3)  PFPS  print  options  with  available  printer. 

1 

2 

3 

4 

5 

6 

7 

Comments/What  could  be  better?: 

4)  PFPS  printed  products  (i.e.,  route  cards, 
maps)  with  briefing/smartpack  requirements. 

Cormnents/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

7 

5)  PFPS  Threat  Menu  selections  with 
mission  planning  requirements. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

7 

6)  PFPS  Aircraft  Weight  &  Performance 

Menu  selections  with  mission  planning 
requirements. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

7 

7)  PFPS  Route  Construction  menu  selections 
with  mission  planning  requirements. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

7 

8)  PFPS  Airspace  Overlay  menu  selections 
with  mission  planning  requirements. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

7 

V.  RELIABILITY 

Using  the  scale  below,  rate  your  opinion  of  the  level  of  reliability  experienced  while  using  PFPS  for  mission  planning 
by  circling  a  coneqjonding  number.  Comments  may  be  made  after  each  question. 

EXTREMELY  NEUTRAL  EXTREMELY 

UNRELIABLE  RELIABLE 

1)  The  CFPS  module.  1  2  3  4  5  6  7 


rrntu  couut  oe  oeueri, 

2)  The  FalconView  module. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

7 

3)  The  Route  Server. 

Comments/What  could  be  better?: 

1 

2 

3 

4 

5 

6 

7 

EXTREMELY 

NEUTRAL 

EXTREMELY 

UNRELIABLE 

RELIABLE 

4)  PFPS  running  on  Windows  95. 

1  2 

3  4  5 

6  7 

Comments/What  could  be  better?: 

• 

5)  While  using  PFPS,  approximately  how  many  times  per  session  did  PFPS  or  one  of  its  modules  crash?  (stop 

working)  _ 

Comments:  _ 

6)  Did  these  crashes  end  your  session  or  were  you  able  to  restart  PFPS  or  the  failed  module(s)  and  continue?  (Circle 
one) 

Yes,  session  ended  No,  session  continued  Mixed  No  Crashes 
Comments:  _ _ 

7)  While  you  were  using  PFPS,  approximately  how  many  times  per  session  did  Windows  95  crash?  (stop  working) 


Comments'.  _ _ 

VL  MAINTAINABILITY 

Using  the  selections  available  for  each  question,  provide  your  answer.  Comments  may  be  made  after  each  question. 

1)  If  the  minimum  required  computer  (100  megahertz,  Pentium  PC,  numing  Windows  95)  was  available  to  your  unit, 
would  you  be  comfortable  installing/reinstalling  PFPS  and  setting  up  the  required  databases  and  modules  for  your 
unit’s  use?  (Circle  one) 

Yes  No  Don’t  Know 

Comments  :  _ _ _ 

2)  With  your  training  in  PFPS  up  to  this  point,  do  you  feel  comfortable  troubleshooting  PFPS  software  malfunctions? 
(Circle  one) 

Yes  No  Don’t  Know 


Comments:  _ _ 

3)  With  your  training  in  PFPS  up  to  this  point,  could  you  locate  information  on  how  to  access  technical  siqiport? 
(Circle  one) 

Yes  No 


Comments  :  ^ _ 

4)  With  your  training  in  PFPS  iq)  to  this  point,  do  you  think  PFPS  requires  a  WTI  to  train  other  unit  personnel  in  its 
use?  (Circle  oneV 

Yes  No  Don’t  Know 


Comments:.  _ _ _ _ _ _ 

VL  GENERAL 

Using  the  scale  below,  rate  the  validity  of  each  proposed  mission  planning  system  ftmctionahty  enhancement  as  is 
concerns  your  conummity.  Comments  on  your  rating  may  be  made  after  each  item. 

INVALID  NEUTRAL  VALID 

1)  Mission  Data  Load  (MDL/Brick)  capability.  1  2  3  4  5  6  7 

Why  did  you  rate  it  this  way?: _ _ _ _ _ 


INVALID 

2)  Aircraft  Specific  Flight  Performance  Modules.  1  2 

(FPMs) 

Why  did  you  rate  it  this  way?: 

3 

NEUTRAL 

4 

5 

6 

VALID 

7 

3)  Takeoff  and  Landing  Data  (TOLD)  1 

weight  and  balance  calculation  capability. 

tVhy  did  you  rate  it  this  way?: 

2 

3 

4 

5 

6 

7 

4)  Hover  Mode  Tool.  1 

Why  did  you  rate  it  this  way?: 

2 

3 

4 

5 

6 

7 

5)  Circumlinear  (serpentine)  route  drawing  tool.  1 

Why  did  you  rate  it  this  way?'. 

2 

3 

4 

5 

6 

7 

6)  Capability  to  scan  in  any  map  and  use  it.  1 

Why  did  you  rate  it  this  way?: 

2 

3 

4 

5 

6 

7 

7)  Capability  to  access  other  than  strictly  1 

NIMA-produced  Compressed  Arc  Digital 

Raster  Graphics(CADRG)  files. 

Why  did  you  rate  it  this  way?: 

2 

3 

4 

5 

6 

7 

8)  You  are  given  a  limited  amount  of  fimding  and  asked  to  rank  the  mission  planning  system  fimctionality 
enhancement  listed  below  according  to  which  you  would  like  to  buy  first  all  the  way  to  seventh.  Please  rank  the 
following  according  to  your  specific  communities  needs,  using  the  numbers  one  through  seven  only  once. 

_ Mission  Data  Load  (MDL/Brick)  capability 

_ Aircraft  Specific  Flight  Performance  Modules  (FPMs) 

_ Takeoff  and  Landing  Data  (TOLD)  weight  and  balance  calculation  capability 

_ Hover  Mode  Tool 

_ Circumlinear  (serpentine)  route  drawing  tool 

_ Capability  to  scan  in  any  map  and  use  it 

_ Capability  to  access  other  than  strictly  NIMA-produced  compressed  digital  graphics 

9)  Are  there  any  other  enhancements  you  would  like  to  see?  Please  do  not  include  them  in  the  above  ranking. 


10)  Using  the  scale  below,  rate  the  level  of  Tactical  Impact  using  PFPS  would  provide  have  on  missions  in  your 
community. 


Tactical  Impact. 


VERY 

INSIGNIFICANT 

NEUTRAL 

VERY 

SIGNmCANl 

1  2 

3  4  5 

6  7 

1 1)  Using  the  scale  below,  rate  the  level  of  Time  Savings  using  PFPS  would  provide  have  on  missions  in  your 
community. 


VERY 

NEUTRAL 

VERY 

INSIGNmCANT 

SIGNinCANl 

Time  Savings. 

1  2 

3 

4  5 

6  7 

Time  savings  in  what 
areas?. 


Thank  You  For  Your  Time 


APPENDIX  D.  OPERATIONAL  RISK  MANAGEMENT  PLAN 


D.1  Operational  Risk  Management  (ORM).  Two  areas  have  been  identified  to  present 
potential  hazard  to  personnel  and  equipment  during  this  assessment.  Each  is  detailed  in 
following  sections,  along  with  pertinent  risk  decisions,  controls  to  be  implemented,  and 
supervisory  roles  designated.  Specific  potential  hazards  include; 

-  WTI  control  points  in  the  PFPS  waypoint  database  possibly  entered  incorrectly. 

-  Weight  and  performance  data  in  the  baseline  aircraft  routes  possibly  entered 
incorrectly. 

D.1.a  Hazard:  WTI  control  points  in  the  PFPS  waypoint  database  possibiy  entered 
incorrectly. 

Assessment:  With  the  introduction  of  products  from  AMPSs  into  the  cockpit,  there  is  an 
inherent  risk  that  the  information  may  be  incorrect  in  the  databases  available  for  user 
selection. 

Risk  Decisionfs):  This  hazard,  given  that  the  controls  and  supervision  is  completed,  is 
determined  to  be  'low  risk'. 

ControKs)  Implemented:  The  WTI  Control  Point  database  will  be  screened  for  accuracy  in 
labeling  and  position. 

Supervisorv  Role(s):  The  Project  Action  Officer  will  ensure  that  each  point  in  the  WTI 
Control  Point  database  is  screened  for  accuracy  in  labeling  and  position  prior  to  its  use  in 
mission  planning. 

D.1.b  Hazard:  Weight  and  performance  data  in  the  baseline  aircraft  routes  possibly 
entered  incorrectly. 

Assessment:  With  the  introduction  of  products  from  AMPSs  into  the  cockpit,  there  is  an 
inherent  risk  that  the  information  may  be  incorrect  in  the  databases  available  for  user 
selection. 

Risk  Decisionfs):  This  hazard,  given  that  the  controls  and  supervision  is  completed,  is 
determined  to  be  'low  risk'. 

ControKs)  Implemented:  The  individual  aircraft  weight  and  performance  data  in  the 
baseline  aircraft  routes  will  be  screened  for  accuracy  in  all  areas. 

Supervisorv  Role(s):  The  division  PFPS  Model  Manager  will  build  and  check  their  specific 
aircraft  weight  and  performance  data  in  the  baseline  aircraft  routes  for  accuracy  in  all 
areas  prior  to  its  use  for  mission  planning. 


APPENDIX  E.  ASSESSMENT  REPORTS/DELIVERABLES  PLAN 


ASSESSMENT  REPORTS/DELIVERABLES  PLAN 


Qualitative  Assessment  Title: 

PORTABLE  FLIGHT  PLANNING  SOFTWARE 
(PFPS) 

Applicable  Requirements  Documents 
(Mission  Needs  Statement,  TEMP,  ORD,  etc.): 

TAMPS  OAG  AFTER-ACTION  MESSAGE 
NAVSTKAIRWARCEN  161540Z  MAY  97 

Sponsor/Tasking  Agency  Representative 
(Rank,  Name,  E-mail,  Phone,  FAX): 

PMA-233,  MAJ  STEVE  BENNETT  USMC 

DSN  664-3426  x7903,  (703)  604-3426 
bennettjs.ntprs@navair.navy.mil 

Project  Action  Officer 

(Rank,  Name,  E-mail,  Phone,  FAX): 

CAPT  ROB  BARR  USMC,  DSN  951-3469, 

(520)  341-3469  barrr@pendleton.usmc.mil 

TAILORED  REQUIREMENTS  LIST: 


DELIVERABLE 

DUE  DATE 

CONFIGURATION  or 
FORMAT 

REMARKS* 

PFPS  TRAINING 

16  JUL  97' 

LIVE  TUTORIAL 

COMPLETE,  LITTON 
PRC 

PFPS  VER  3.0 

01  SEP  97 

COMPACT  DISC 

FROM  PMA-233  TO 
MAWTS-1. 

rWO  (2)  200  MHZ 
ICOMPUTERS 

01  SEP  97 

CPU,  MONITOR, 
MOUSE,  KEYBOARD, 
REQUIRED  CABLES, 
WIN  95/NT 

INSTALLED 

FROM  PMA-233  TO 
MAWTS-1. 

PFPS  VERX.X 

WHEN  AVAILABLE 

COMPACT  DISC 

FROM  PMA-233  TO 
MAWTS-1. 

CONTINUED  EVAL 

INDEFINITE 

NA 

FROM  MAWTS-1  TO 
PMA-233 

FINAL  REPORT 

05  JAN  98 

ELECTRONIC/ 

HARDCOPY 

FROM  MAWTS-1  TO 
PMA-233 

*lndud*  who  the  deliverable  will  come  from  and  who  the  deliverable  will  go  to. 


DATE  APPROVED: 

FOR  MAWTS-1 :  ^ - - 

FOR  SPONSOR/TASKING  AGENCY:  ^  - 


4 


COLOPHON 


This  report  was  word  processed  using  Microsoft  Word  97.  The  body  text  font  is  Anal  1 1 
point  pitch;  figure  and  table  description  font  is  Anal.  10  point  pitch;  and  the  survey  font  is  Times 
New  Roman,  10  point  pitch.  All  data  was  analyzed  using  Microsoft  Excel  97  and  all  charts  in  the 
results  section  were  made  using  the  Chart  Wizard  in  Excel. 

The  charts  in  section  3.0,  Description  of  Software,  were  produced  in  Microsoft  PowerPoint 
S.aand  were  provided  courtesy  of  PMA-233.  Updates  to  the  charts  were  made  in  Powerpoint 

Images  of  the  Microsoft  Windows  95  and  PFPS  Desktop  were  made  using  the  ‘Print  Screen’ 
function  and  were  saved  as  bitmaps  for  import  into  the  document  using  Microsoft  Paint. 


